crawl4ai

🚀🤖 Crawl4AI: the open-source web crawler for LLMs and AI agents

Latest: [v0.9.5](https://github.com/unclecode/crawl4ai/releases/tag/v0.9.5) (14 Oct 2026) · all releases →

Crawl4AI turns any website into clean, LLM-ready Markdown for RAG, AI agents and data pipelines. Run the open-source web crawler and scraper yourself, free forever, or use it hosted with one key: /scrape a page to Markdown, /extract it to JSON, /search the web, /serp for Google's results page as data, /answer a question with sources; MCP for your agent. Free: 1,000 credits every month, 10 a minute, 2 at a time. Paid: 120 a minute, 10 at a time, from $10 a month.

Two ways to use Crawl4AI

🐍 Run it yourself: open source, forever

pip install -U crawl4ai
crawl4ai-setup        # installs the browser, once

import asyncio
from crawl4ai import AsyncWebCrawler

async def main():
    async with AsyncWebCrawler() as crawler:
        result = await crawler.arun(url="https://news.ycombinator.com")
        print(result.markdown)

asyncio.run(main())

Docker server, CLI and every option: Installation · docs.crawl4ai.com

☁️ Or use the cloud: no browsers, no proxies

1. 

Verify your email and 1,000 free credits are yours every month. No card.

1. Get any page as Markdown:

   curl -s https://api.crawl4ai.com/scrape \
     -H "Authorization: Bearer $CRAWL4AI_KEY" \
     -H "Content-Type: application/json" \
     -d '{"url": "https://news.ycombinator.com"}' | jq -r .markdown

The same key works for /extract, /search, /serp, /answer and many URLs at once (/scrape/batch, /scrape/jobs). Pay as you go: live prices.