live · self-serve keys + dashboard

Crawl, scrape, and audit any website.

One unit of pricing: the credit. 1 credit = 1 chargeable page (scan, scrape, crawled page, or actor call). Flat $0.001 per credit on every monthly plan — bigger plans never cost less per credit. Eight specialty actors included: render-diff, sitemap-audit, internal-link-graph, structured-data and more.

Start free → Read the docs

Free: 1,000 credits / mo, no card. Plans from $5 / 5K credits to $100 / 100K credits. See the full pricing →

200 OK

$curl -X POST https://api.crawlcrawl.com/v1/scan \
  -H "Authorization: Bearer crk_..." \
  -d '{"url":"https://example.com"}'

{
  "url": "https://example.com",
  "status": 200,
  "markdown": "# Example Domain...",
  "ai_bot_policy": { "GPTBot": "allowed", ... }
}
✓ 1 page billed

$curl -X POST https://api.crawlcrawl.com/v1/crawls \
  -d '{"url":"https://docs.example.com","max_pages":500}'

{ "id": 1234, "status": "queued" }

# 30s later, webhook fires with the dataset:
{ "crawl_id": 1234, "pages": 487, "errors": 0 }
✓ 487 pages billed

$curl -X POST https://api.crawlcrawl.com/v1/actors/audit-onpage \
  -d '{"url":"https://your-client.com"}'

{
  "summary": { "errors": 1, "warnings": 3 },
  "findings": [
    {"rule":"h1.missing", "severity":"error"},
    {"rule":"meta.description-too-long", "count":207}
  ]
}
✓ 1 credit billed

$curl -X POST https://api.crawlcrawl.com/v1/actors/render-diff \
  -d '{"url":"https://your-spa.com"}'

{
  "static":   { "word_count": 12,   "link_count": 0 },
  "rendered": { "word_count": 2482, "link_count": 39 },
  "ai_bot_blind_pct": 0.995,
  "assessment": "severe — most content invisible to AI"
}
✓ 2 credits billed (1 page + 1 cloud-page)

/ why this one

Three reasons teams pick it.

/ predictable pricing

One credit per page

Same per-credit price on every monthly plan. Bigger plans never cost less per credit. You always know what next month costs.

/ eight specialty endpoints

POSTs you'd otherwise glue together

Render-diff, sitemap-audit, internal-link-graph, structured-data, on-page audit, article extraction, broken-link check, llms.txt builder. Each is one POST, one credit, on every plan including Free.

/ drop-in for Firecrawl

Same shapes, swap the base URL

/v1/scrape, /v1/map, and /v1/crawl return the same envelope shape your existing parser already handles. Change two strings, redeploy.

/ how it works

Three integration paths. One API.

Drop in one endpoint, or wire the whole platform. Same JSON shape, same billing counter, same auth.

/ for developers

Single-URL scrape

POST a URL, get clean markdown plus structured signals back in one round-trip. Pair with the eight actor endpoints for technical SEO and AEO audits. /v1/scan →

/ for RAG pipelines

Multi-page crawl

Queue a crawl, get a webhook when it finishes with a paginated dataset of clean markdown. Schedule recurring monitors with diff-only delivery. /v1/crawls →

/ for agencies

SEO + AEO actors

Eight specialty endpoints for the work most teams used to glue together — audit-onpage, render-diff, sitemap-audit, internal-link-graph and more. See all actors →

/ actors

Eight specialty endpoints. Each one POST, each one credit.

No dashboard, no queue, no waiting room. Send a URL or a crawl id, get the answer back as JSON.

/v1/actors/audit-onpage

On-page audit

About 30 technical SEO rules per page. Flags missing h1, meta length, robots conflicts, canonical loops, hreflang mismatches.

/v1/actors/extract-article

Article extraction

Clean article body with author and date. Strips navigation, footer, comment threads, and ad slots automatically.

/v1/actors/check-links

Broken-link check

Validate every link on a page. Optional anti-bot retry rescues LinkedIn 999 and Cloudflare false positives.

/v1/actors/structured-data

Structured data

JSON-LD, Microdata, RDFa, OpenGraph, Dublin Core, Microformats — six syntaxes parsed and merged into one JSON.

/v1/actors/render-diff

Render diff

Static fetch versus full browser render. Returns ai_bot_blind_pct — the share of the page invisible to GPT-class crawlers.

/v1/actors/internal-link-graph

Internal link graph

PageRank, weakly-connected components, and orphan detection on any crawled site. Runs on an existing crawl id.

/v1/actors/sitemap-audit

Sitemap audit

Seven-bucket health: ok, redirect, 4xx, 5xx, noindex, canonicalised away, network error. dry_run is free.

/v1/cloud/proxy-fetch

Anti-bot fetch

4 pools across 190+ countries. Bytes-priced; cheaper than a full Chrome render when you don't need JavaScript.

All eight actors are included on every paid plan — and on the free tier. Full API reference →

/ migrating

Already coding against Firecrawl? Swap the base URL.

Three drop-in aliases. Each returns the same envelope shape your existing parser already handles. Change the base URL, change the API key prefix, redeploy.

POST /v1/scrape

scrape

Returns { success, data: { markdown, html, metadata } } — the same shape your existing Firecrawl client parses. No client-side changes required.

POST /v1/map

map

Returns { success, links: [...] } — every URL discoverable from one entry point.

POST /v1/crawl

crawl

Async crawl with the same webhook payload format. Returns a job id; webhook fires when the dataset is ready.

/v1/extract (LLM-driven extraction) is not yet shipped — the endpoint returns 501 with a breadcrumb pointing at /v1/actors/structured-data for deterministic schema.org extraction. Full migration notes →

/ what's in the box

Every feature, at every paid tier. No add-ons.

JavaScript rendering: Real Chrome render for SPAs and hydration-heavy pages. Auto-fallback on direct-fetch failures. Same per-page price.
Anti-bot routing: Bypass Cloudflare, Akamai, PerimeterX, Datadome, captchas across 190+ countries. One endpoint, no configuration.
Scheduled crawls + monitors: Cron syntax, HMAC-signed webhooks. Fire only on real content changes via the built-in diff endpoint.
Dataset storage: Every crawl's pages are stored and paginate-able indefinitely. No expiry. Re-pull historical datasets at zero credit cost.
Structured data + AI-bot policy: Schema.org, OpenGraph, JSON-LD, hreflang, canonical, AI-bot robots resolution — included on every scrape response.
Permanent HTML archive: Every fetched page's raw HTML is stored long-term alongside the structured output. Re-pull historical datasets years later at zero credit cost via GET /v1/crawls/{id}/export. Retention policy.
llms.txt builder: Generate a publishable /llms.txt for any site in one call. Single endpoint, no external tool required.
API key rotation + audit log: Per-key rotation, soft-revoke, last-used timestamps. Every chargeable call writes an append-only ledger entry.

/ pricing

$0.001 per credit. Flat rate. Every tier.

1 credit = 1 chargeable page (scan, scrape, crawled page, or actor call). Same per-credit price on every monthly plan — bigger plans never cost less per credit. Free to start, no card.

Hobby

$5/mo

For personal projects and weekend builds.

5,000 credits / month
4 concurrent requests
All 8 actor endpoints included
Anti-bot routing, monitors, permanent archive
Community support

Start Hobby →

Starter

$10/mo

For early-stage startups and solo founders.

10,000 credits / month
8 concurrent requests
All 8 actor endpoints included
Email support
HMAC-signed webhooks + monitors

Start Starter →

Things people ask before signing up.

Is there a free tier?: Yes. 1,000 credits per month, no card. All eight actor endpoints included. Permanent — not a 14-day trial.
What counts as a credit?: One credit = one chargeable call. /v1/scan is 1 credit. A 100-page result from /v1/crawls is 100 credits. Each actor invocation is 1 credit. Read-only intelligence calls — link graph, orphans, robots policy — are free.
What latency should I expect?: Reads from already-stored data — page content, link graph, orphans — return in under a second. A single-URL scrape is bounded by the target site's response time: typically one to three seconds for a typical page.
Can I migrate from Firecrawl?: Yes. Change the base URL to https://api.crawlcrawl.com and the API key prefix to crk_. /v1/scrape, /v1/map, and /v1/crawl return the response shapes your existing parser already handles.
Do unused credits roll over?: Monthly-plan credits reset at the start of each billing cycle.

Free to start.

1,000 credits per month, no card required. All eight actor endpoints. One credit per page on every paid plan.

Start free → Read the docs