unclecode/crawl4ai
Crawl4AI turns browser-rendered pages into Markdown or structured data for agent and RAG pipelines. v0.9.2 fixes streaming leaks and deployment issues after two security-focused releases.
Crawl4AI provides async browser crawling, Markdown generation, schema-based extraction, sessions, proxies, caching, deep crawls, and an MCP connection. **v0.9.2** fixes a streaming task/page leak plus Docker auth, packaging, and GPU-build issues.
Builders using web retrieval should update rather than pinning older Docker images, keep API authentication enabled, and treat crawl requests as untrusted input. **v0.9.0** made the Docker server secure by default, while crash recovery and prefetch support help longer jobs.
Crawl4AI provides async browser crawling, Markdown generation, schema-based extraction, sessions, proxies, caching, deep crawls, and an MCP connection. **v0.9.2** fixes a streaming task/page leak plus Docker auth, packaging, and GPU-build issues. Builders using web retrieval should update rather than pinning older Docker images, keep API authentication enabled, and treat crawl requests as untrusted input. **v0.9.0** made the Docker server secure by default, while crash recovery and prefetch support help longer jobs. The broad feature set increases deployment and browser-state complexity. **v0.8.7** fixed critical RCE, SSRF, auth-bypass, file-write, XSS, and JWT-secret problems, so exposed legacy servers warrant particular scrutiny; the synchronous Selenium path is also deprecated.
Crawl4AI adds a concrete, feature-rich browser retrieval backend to the MCP tool landscape, but its release history makes security patching and isolation part of the architecture rather than routine maintenance. It can supply structured web material to larger context systems, while recent leak and severe vulnerability fixes narrow the acceptable deployment path to current, authenticated versions with untrusted requests contained.