For the complete Lumen Crawler documentation index, see llms.txt. A markdown version of any docs page is available by appending .md to its URL. Docs MCP: /docs/mcp.
Auth and keys
Bearer API keys for crawls. Cognito JWT for account and billing.
Crawl plane
Every /v1/crawls* request (except the console list fallback) needs:
Authorization: Bearer lc_live_…Production AUTH_MODE is registered. Unknown or revoked keys return 401 unregistered_key. Missing or malformed Bearer returns 401 unauthorized.
Keys are created in the console. Lookup is SHA-256 of the full secret. Only a 12-character prefix is stored for display.
Account plane
/v1/account* and /v1/billing* use the Cognito ID token from the console session (Authorization: Bearer <jwt>).
GET /v1/crawls accepts either a crawl key or that JWT so the console can list crawls for the account.
CORS
The API allows Authorization, Content-Type, and Idempotency-Key from https://crawlapp.lumen-labs.ai and http://localhost:3000.
Last updated on