Rate limits & quotas.

Three caps

Every project has three hard limits:

• Daily pages: pages crawled per day, resetting at UTC midnight.
• Monthly pages: pages crawled per month, resetting at UTC month start.
• Concurrent runs: total queued and running crawls.

/v1/cloud/* endpoints count against the daily pages cap.

Hitting a cap

When a cap is reached, the API returns HTTP 429 with a Retry-After header in seconds.

HTTP/1.1 429 Too Many Requests
Retry-After: 3600
Content-Type: application/json

{
  "error": {
    "code": "rate_limit_exceeded",
    "message": "Daily page cap reached. Retry after 3600 seconds."
  }
}

Check usage

Call GET /v1/usage to see current counters, caps, and percent used.

curl https://api.crawlcrawl.com/v1/usage \
  -H 'Authorization: Bearer crk_...'

{
  "daily_pages": {
    "used": 87,
    "cap": 100,
    "percent": 87
  },
  "monthly_pages": {
    "used": 870,
    "cap": 1000,
    "percent": 87
  },
  "concurrent_runs": {
    "used": 0,
    "cap": 1
  }
}

Tier limits

Tier	Daily pages	Monthly pages	Concurrent runs
Free	100	1,000	1
Indie	1,000	10,000	5
Agency	5,000	50,000	10
Studio	10,000	100,000	20

Exceeding limits

Contact support before hitting monthly caps to discuss upgrades. Mid-month upgrades are prorated.