Crawl4AI hosting

A crawler that returns clean Markdown instead of raw HTML.

Search & web agentsApache-2.0from $11.99/mo

In short

Crawl4AI hosting at AIHostBox means a dedicated, single-tenant Crawl4AI instance on our infrastructure: we install it, pin it to a reviewed release, put it behind TLS on your domain, back it up nightly and keep it patched. Plans start at $11.99 per month and the admin account, the data and any API keys are yours.

Markdown outputJS renderingStructured extractionAsync

Crawl4AI is an Apache-2.0 crawler written in Python by UncleCode and a sizeable contributor community. Scrapers usually hand you HTML to clean up; Crawl4AI drives a real Chromium browser through Playwright, waits for JavaScript and lazy-loaded content, and returns Markdown shaped for language models, with optional BM25 or heuristic filtering to keep only the useful part of a page. It can also extract JSON with CSS or XPath schemas, or with an LLM when the structure is irregular.

Its Docker server wraps all this in a REST API with a browser pool, deep crawling (breadth- or depth-first), screenshots, PDF output and a built-in playground. Browsers are memory-hungry and long crawls take hours, which is why it belongs on a server: your pipeline calls one URL, the crawl carries on overnight, and your laptop is not the thing holding forty tabs open. It also gives the crawler one stable IP address and a fixed browser version, so results do not change with whoever ran the job.

Crawl4AI hosting plans

PlusRecommended minimum

4 GB RAM · 2 vCPU · NVMe

$11.99/mo

Order

ProTeams & heavier load

8 GB RAM · 3 vCPU · NVMe

$18.99/mo

Order

Every plan is a dedicated single-tenant instance with NVMe storage, nightly off-box backups, TLS and managed patching. Quarterly, semi-annual and annual terms are discounted at checkout. Model tokens are not included — you bring your own key.

How we run Crawl4AI

The official unclecode/crawl4ai container: a Gunicorn/FastAPI server on port 11235 with a pool of headless Chromium browsers, 1 GB of shared memory for the browsers and an internal Redis for jobs and rate limiting.

What we configure for you

CRAWL4AI_API_TOKEN set so the API and MCP endpoints require a bearer token
LLM_PROVIDER and the matching provider key in .llm.env, only if you use LLM extraction
config.yml memory threshold and browser pool sized to the plan’s RAM
Rate limiting (rate_limiting.default_limit) set for your expected traffic
Prometheus metrics and the /monitor dashboard left on so you can see browser usage

Which plan fits

Plus (4 GB) matches the project’s own container limit and runs a small browser pool for documentation-site crawls and scheduled jobs. Pro (8 GB) allows more concurrent browsers for large deep crawls, screenshot or PDF batches, and several pipelines calling it at once.

The honest limits

Headless Chromium is the constraint: each browser takes hundreds of megabytes, so concurrency is bounded by RAM rather than bandwidth. LLM extraction is only as quick as the model behind it, and on a CPU-hosted local model it is slow.

What people run Crawl4AI for

Feed a RAG index

Crawl a documentation site into clean chunks.

Monitor sources

Scheduled extraction into your own store.

Structured product data

Pull prices, specs or listings into JSON with a CSS schema and no model calls.

Web tool for agents

Give an agent a crawl tool through Crawl4AI’s MCP endpoint.

Licence and version policy

Item	Position
Upstream licence	Apache-2.0
Sold as managed hosting	Yes
Tenancy	Single tenant — one customer per instance
Patching	Critical advisories within 24 hours, high severity within 72

Our full position on licences, including the applications we decline to sell, is on the licensing page.

Works well with Crawl4AI

More Search & web agents we host

Questions

Crawl4AI hosting: your questions

How do I call it?

Send a POST to /crawl with a list of URLs and a crawler config, or use /md for Markdown only; /crawl/stream returns results as they finish and /crawl/job queues long work. There is also /screenshot, /pdf and /execute_js. Every request carries your token as an Authorization: Bearer header.

Does it need an LLM key?

No. Crawling, Markdown generation and CSS or XPath extraction run without any model. Only LLM-based extraction calls a model, which you set with LLM_PROVIDER and the matching key such as OPENAI_API_KEY or ANTHROPIC_API_KEY, or LLM_BASE_URL for an OpenAI-compatible server you host.

Can AI agents use it directly?

Yes. The server exposes Model Context Protocol endpoints over SSE and WebSocket, so MCP clients such as Claude Code or Cursor can call its crawl, Markdown and screenshot tools. The same bearer token protects them.

How is the server secured?

Since 0.9.0 the Docker server refuses to listen on published ports without CRAWL4AI_API_TOKEN, and it only allows declarative hooks rather than inline Python. We set a strong token at setup and serve the API over TLS; the /health check is the only unauthenticated route.

Will it get past bot protection and logins?

It can keep persistent browser profiles and sessions, use proxies, and has a stealth mode, which handles many ordinary sites. It does not solve CAPTCHAs, and you remain responsible for respecting a site’s terms and robots rules. Heavily protected sites may still block a datacentre IP address.

Run Crawl4AI without running a server

A dedicated Crawl4AI instance, deployed and maintained by us. Month to month, cancel whenever.

Browse the catalogue Talk to a human