For the complete Lumen Crawler documentation index, see llms.txt. A markdown version of any docs page is available by appending .md to its URL. Docs MCP: /docs/mcp.
Plans and quota
Monthly included pages, create-time 402, and what counts as a page.
Usage is measured in successful pages (pages_ok, HTTP 2xx/3xx) per UTC calendar month. Robots-skipped URLs and failed fetches increment counts.pages but do not consume quota.
Plans
| Plan | Price | Included pages / UTC month |
|---|---|---|
| Free | $0 | 100 |
| Starter | $29 | 2,000 |
| Pro | $99 | 15,000 |
Upgrade and manage billing in the console. Checkout is Stripe hosted. Billing is currently in Stripe test mode.
Paid overage (Starter 0.015 / page) is only billed when overage metering is enabled for the account. Otherwise paid plans hard-cap like Free.
Create-time reservation
POST /v1/crawls checks that max_pages fits remaining included pages. Free with 0 used and max_pages: 101 returns 402 quota_exceeded even if the site is smaller.
plan_inactive (402) means the account is past_due or canceled.
If a running crawl hits the remaining budget, stop_reason is quota_exhausted.
GET /v1/account/usage (console JWT) returns plan, included, used, remaining, period_end, overage_pages. period_end is the first instant of the next UTC month.
For agents
On 402, inspect usage. Lower max_pages or wait for the next UTC month. Do not retry the same max_pages. Failed fetches do not consume quota.
Last updated on