Search.

Why teams need a search API

Most AI applications eventually hit a wall where their training data is no longer fresh enough. The model can answer general questions, but for current events, recent product releases, or anything that happened after the training cutoff, the answers degrade quickly. The standard solution is to ground the model in live search results: ask the search API for current information, feed the results into the prompt, and let the model synthesize a grounded answer.

That pattern is now standard across AI assistants, agent products, and retrieval-augmented-generation pipelines. The challenge is operational. Running your own SERP-scraping infrastructure is fragile: search engines change their rendering, anti-bot protection escalates, and the engineering hours add up. A managed search API removes that pile of glue code from your roadmap.

Web search in one call

Fresh results, the same shape every other crawlcrawl endpoint returns. One POST, structured JSON back.

curl -X POST https://api.crawlcrawl.com/v1/cloud/search \
  -H "Authorization: Bearer crk_..." \
  -d '{"query":"latest llm benchmarks 2026","num":10}'

# returns
{
  "items": [
    { "url": "...", "title": "...", "content": "...snippet...", "status": 200 },
    ...
  ],
  "result_count": 10
}

Each result includes the URL, title, and a content snippet. For deeper context, follow the URLs into the regular /v1/cloud/scrape or /v1/crawls endpoint to fetch full page content with structured-data signals, all under the same API key.

Outbound link extraction

For mapping the link graph out of a target page (useful for crawl seeding, broken-link audits, citation discovery) use POST /v1/cloud/links. The endpoint returns URLs found on the input page along with their anchor text. This is not a backlink-intelligence tool; to find who links to a target across the open web you need a dedicated index like Ahrefs, Semrush, or Moz. For outbound mapping inside a known starting point, it is the right primitive.

Common use cases

Three patterns explain most search-API usage on crawlcrawl.

The first is AI agent grounding. An agent receives a user question that requires current information ("what is the latest price of X stock", "what did the SEC announce yesterday", "who released a new model this week"). The agent calls the search API, retrieves a handful of recent results, and synthesizes a grounded answer with citations.

The second is RAG with live search backstop. A retrieval-augmented-generation product uses its vector database for cached knowledge and the search API for queries the database does not cover well. The architecture stays simple because one provider covers both the corpus crawl and the live search.

The third is competitive and market intelligence. A team monitors specific queries (their brand, their competitors, key category terms) on a scheduled basis. New results trigger alerts. Pair the search endpoint with the monitor primitive to make this trivial to operate.

Pricing for search

Each search request consumes credits at the same per-request rate as a scrape. Search results returned do not multiply the cost; one query at num: 10 is one credited request. The free tier covers 1,000 credits per month with no card; Pro at $8/mo covers 5,000 pages, enough for a small AI agent in production. Higher tiers scale the credit pool for larger volumes. See full pricing →

Where it fits in the platform