Unblock API — Bypass Cloudflare, Akamai, Captchas in One Call

When to reach for unblock specifically

The unblock endpoint exists for a very specific moment: you have a working scraper, you have a working pipeline, and a single class of targets is breaking it with a challenge page or a captcha. You do not want to rip out what you have and adopt a new platform. You want a drop-in unblock primitive that takes a URL and returns the resolved page body.

That is what POST /v1/cloud/unblock is. It is not a crawler, not a full-platform replacement, not an opinionated framework. It is the narrow function of "make this URL retrievable" exposed as a stateless REST call. Send a URL, receive content. Bill by credit.

What it bypasses

The endpoint is designed to handle the bot-defence systems that most often interrupt production scraping:

Cloudflare bot management — the challenge interstitial, JavaScript challenges, managed challenge pages.
Akamai Bot Manager — sensor data, behavioural fingerprinting, the AKAM cookie flow.
PerimeterX — the px-captcha challenge and device-fingerprinting check.
Datadome — the dd cookie flow and challenge resolution.
reCAPTCHA v2 / v3 — resolved transparently when the challenge appears in the request flow.
hCaptcha — resolved through the same routing path.
Generic rate-limit pushback — 429 responses with backoff and retry handled by the routing layer.

How a request looks

The endpoint takes a URL and a return format. That is the minimum viable call. Optional parameters let you target the request to a specific country, return raw HTML instead of markdown, or include the response headers.

curl -X POST https://api.crawlcrawl.com/v1/cloud/unblock \
  -H "Authorization: Bearer crk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://target.example/page-behind-cloudflare",
    "return_format": "markdown",
    "country": "us"
  }'

# returns
{
  "url": "https://target.example/page-behind-cloudflare",
  "status": 200,
  "content": "# Page title\n\nClean markdown body here...",
  "cost_usd": 0.0012,
  "elapsed_ms": 4180
}

The country field is optional. When omitted, the routing layer picks the country closest to the target's expected audience. When set, every request goes through that country's residential pool. Useful when you need to see what a customer in a specific region would see (geo-priced pricing pages, region-locked content, country-specific availability).

Drop-in for an existing scraper

If you already have a working scraper and only need an unblock fallback for a handful of stubborn targets, the smallest possible integration is to call unblock only when your normal request fails:

# pseudocode
resp = requests.get(url, headers=ua_headers, timeout=20)
if resp.status_code in (403, 429, 503) or "cf-chl" in resp.text:
    # fall back to unblock
    resp = call_crawlcrawl_unblock(url)
return resp.text

This pattern keeps your credit consumption low: your normal scraper handles 95% of traffic at zero crawlcrawl cost, and only the protected targets that genuinely need the heavier path consume credits. For multi-page workloads where almost every target is protected, use POST /v1/cloud/crawl directly — it routes every page through the anti-bot infrastructure without per-request decisions.

Return formats

The endpoint returns the resolved page in your chosen format. Three formats cover most workloads:

markdown — boilerplate stripped, headings preserved. The format most RAG pipelines want.
raw — post-resolution HTML. Use when you have your own parser downstream.
text — plain-text extraction. Useful for full-text search indexing.

If you need a screenshot of the resolved page (for visual diff, archival, or compliance), the related render endpoint returns PNG bytes through the same routing layer.

Pricing

Unblock counts as a credit-weighted request. The credit multiplier depends on the target's protection level (a Cloudflare-only target costs less than a Datadome-protected page with reCAPTCHA), and the actual cost for each request is returned in the cost_usd field of the response. There is no separate "unblock plan." All paid tiers from $8/mo include the unblock endpoint at the standard credit pool.

For most teams, the workload-comparison math against single-purpose unblock APIs comes out in our favour because you get the crawler, scrape, search, monitor, transform, and render endpoints on the same credit pool. See full pricing →

When unblock is not the right tool

Three cases where a different endpoint is a better fit.

If you need to crawl an entire site (many URLs from the same host with link discovery) where every target is protected, use POST /v1/cloud/crawl. For open sites, use POST /v1/crawls on the owned-infrastructure path. Either way, the crawler handles orchestration, pagination, and link extraction in one call.

If you need post-JavaScript HTML from a single-page app where bot detection is not the issue (the site is open, but content only appears after hydration), use POST /v1/cloud/render. Render runs the page in a real browser without the anti-bot routing overhead.

If you have raw HTML or PDF bytes already in hand and only want the formatting cleanup, use POST /v1/cloud/transform. No fetch happens; you supply the content.

Unblock API.