PixelRAG is the official codebase for *[PixelRAG: Web Screenshots Beat Text for Retrieval-Augmented Generation](https://arxiv.org/abs/2606.28344)*, submitted to arXiv on 1 June 2026 by a team from **Berkeley Sky Computing Lab, BAIR, and the Berkeley NLP Group** — Yichuan Wang and Zhifei Li as joint first authors, advised by Matei Zaharia, Joseph E. Gonzalez, and Sewon Min. Released under Apache-2.0, it has gathered roughly 8,704 stars and 733 forks since opening in late May 2026, making it one of the fastest-rising retrieval projects of the year. ## The Argument The paper's premise is stated plainly: *the web is not natively textual.* Every production RAG stack begins by linearising HTML into text, and that step discards layout, visual structure, and formatting. A table becomes a run of tokens with no columns. A chart becomes an alt-text stub or nothing at all. An infographic becomes noise. The retrieval system then searches over an artifact that has already lost the information the user was asking about. PixelRAG's answer is to delete the parsing stage entirely. Documents — web pages, PDFs, images — are **rendered to screenshots**, and both retrieval and reading happen **in pixel space**. The reader model looks at the tile and reads the number off the table, the same way a person would. Two components make it work. The first is rendering: `pixelshot` drives headless Chromium over CDP (or poppler for PDFs) to produce screenshot tiles. The second is a **`Qwen3-VL-Embedding-2B` model, LoRA-fine-tuned on screenshot data with contrastive learning**, that embeds page images into a space where visual content is actually retrievable. ## The Numbers The paper reports up to an **18.1% accuracy improvement** over text-based retrieval baselines, and — the more surprising result — a **3x token cost reduction** through image compression while holding accuracy. That second figure inverts the usual intuition that images are the expensive modality; a compressed screenshot tile carrying a full table costs less than the token sprawl of that table linearised into prose. Gains are reported across a deliberately mixed benchmark set: **NQ** and **SimpleQA** (text-centric QA, where a pixel approach has the most to prove), **MMSearch** (multimodal open-domain QA), **LiveVQA** (noisy news corpora), and **MoNaCo** (agentic). Winning on text-centric NQ is the load-bearing result — it is the benchmark where "just parse the HTML" should be sufficient. Scale is the other claim. The system indexes the **complete Wikipedia corpus as 30 million screenshot images**, and a pre-built index of **8.28M Wikipedia pages** is served live at `api.pixelrag.ai` with no API key and no setup: ```bash curl -X POST https://api.pixelrag.ai/search \ -H "Content-Type: application/json" \ -d '{"queries": [{"text": "What is the capital of France?"}], "n_docs": 5}' ``` The endpoint also accepts an image as the query, which text RAG structurally cannot do. ## Pipeline and Packaging The project is split into installable stages rather than shipped as a monolith, so you pay only for what you run: | Command | What it does | Install | |---|---|---| | `pixelshot` | Document to image tiles (Playwright CDP, PDF) | `pip install pixelrag` | | `pixelrag chunk` / `embed` / `build-index` | Tiles to vectors to FAISS index | `pip install 'pixelrag[embed]'` | | `pixelrag index` | Orchestrates source → ingest → embed → index | `pip install 'pixelrag[index]'` | | `pixelrag serve` | FAISS search API over FastAPI, CPU or GPU | `pip install 'pixelrag[serve]'` | FAISS is the default for local indexes; **Qdrant** is supported as an alternative backend for configurable quantization, disk-backed vectors, payload filtering, and one collection shared across multiple servers. Indexing runs on Linux with CUDA and on macOS with Apple Silicon MPS — `device: auto` picks the backend — and the README includes a no-GPU walkthrough that indexes a single PDF in roughly three minutes on an M-series Mac. Training is quarantined in a separate uv project under `train/` with its own pinned environment (`torch==2.9.1+cu129`, `transformers==4.57.1`), which is the correct way to keep a CUDA-pinned training stack from contaminating an inference install. The LoRA adapters, the full training set, and the data curation pipeline — LLM-augmented query generation, filtering, hard-negative mining — are all published, so adapting a different backbone does not require reverse-engineering the recipe. ## Giving an Agent Eyes The renderer also ships as a **Claude Code plugin** called `pixelbrowse`. Rather than fetching raw HTML, the agent screenshots a page with `pixelshot` and reads the image — so it sees charts, diagrams, tables, and layout as rendered. There is no MCP server and no backend; the skill simply shells out to a local CLI: ```bash claude plugin marketplace add StarTrail-org/PixelRAG claude plugin install pixelbrowse@pixelrag-plugins ``` This is arguably the most immediately useful part of the repository for developers who will never build an index, and it addresses a real failure mode — agents confidently summarising pages whose actual content lives in a chart they never saw. ## Costs and Caveats Pixel-native retrieval is not free. The pre-built base Wikipedia index is a **~217 GB** download, which puts self-hosting the full corpus out of reach for most individuals and makes the hosted API a practical dependency rather than a convenience. Embedding at scale wants a GPU; the CPU/MPS path is for evaluation, not for building a 30M-image index. Storing screenshots is heavier than storing text, and rendering adds a browser to the ingestion path — the bundled turbo `headless_shell` auto-installs on linux-x64 only, with system Chrome detection everywhere else. There are 14 open issues and the project is at v0.4.0, so the API surface should be expected to move. ## Verdict PixelRAG is the strongest published argument yet that HTML parsing is a lossy legacy step in the RAG pipeline rather than a necessary one, and it is backed by a real paper, real benchmarks across five datasets, a live public index, and an Apache-2.0 codebase with the training data attached. Teams whose corpora are visually dense — financial filings, scientific PDFs, dashboards, product documentation — should evaluate it directly against their existing parser. Teams with clean prose corpora will find the storage and rendering overhead hard to justify, and should instead take the `pixelbrowse` plugin and leave the index alone.