Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
Firecrawl is an open-source Web Data API that converts websites into LLM-ready formats including clean markdown, structured JSON, and screenshots. With over 105,000 GitHub stars, it has become a critical piece of AI infrastructure for powering agents with reliable web data. The platform handles JavaScript-heavy pages, proxy rotation, rate limiting, and blocked content transparently. Key capabilities include URL scraping with structured output via Pydantic schemas, AI-powered autonomous web browsing (Agent mode), full website crawling, interactive page actions (click, scroll, type), URL discovery (Map), and asynchronous batch processing of thousands of URLs. Firecrawl integrates with AI tools through the Model Context Protocol (MCP), enabling direct connection with Claude Code, OpenCode, and other AI coding assistants. The platform reports 96% web coverage and P95 latency of 3.4 seconds across millions of pages. Two agent model tiers are available: spark-1-mini for general tasks at 60% lower cost, and spark-1-pro for complex research. Firecrawl is available both as a self-hosted solution and managed cloud service, with SDKs for Python and Node.js, a REST API, and CLI tools.