claude-real-video (crv on PyPI) is a local preprocessing tool that turns a video into something a text-and-image LLM can actually read: the frames that matter, deduplicated, plus a timestamped transcript and a manifest. Point it at a YouTube link or a local file, and it writes crv-out/frames/*.jpg, frames.json, transcript.txt/.json and MANIFEST.txt — then you paste that into Claude, ChatGPT or Gemini and ask questions.

The repository was created on 30 June 2026 and has reached roughly 1,951 stars and 163 forks, with the latest release (v0.7.16) on 21 July and commits through 3 August. It is MIT-licensed with 5 contributors, and reached the Hacker News front page.

The Problem It Targets

Most "let an LLM watch a video" paths do something weaker than they appear to. Paste a YouTube link into ChatGPT and it reads the transcript, not the picture. Claude does not accept a video file at all. Gemini can read video natively, but it uploads the file and samples frames at a fixed interval — 1 fps by default — which over-samples a static screencast and under-samples a fast-cut reel.

crv replaces the fixed quota with scene-change detection plus a density floor, then removes near-duplicates. On a 58-second clip, 1 fps sampling yields 58 frames; crv keeps the 26 that actually differ.

Pipeline

The dedup design is where the engineering sits. A global channel measures real pixel difference on downscaled RGB rather than a perceptual hash — the README notes hashes go blind on flat colours and equal-luma hue changes. A settled-local channel catches what the global one averages away: thin pen strokes, caption swaps and small UI updates, gated so it only fires on a settled new state rather than motion mid-flight, with 1px shift tolerance so grain and jitter do not trigger it. Because the window spans the last N kept frames rather than only the previous one, an A-B-A cutaway does not re-send a shot the model has already seen.

Version 0.7.16 added a third "action" channel after a structural blind spot surfaced in a user's 2,181-video batch run: a percentage comparator can never detect a subject covering under 1% of the frame, because it cannot change 8% of the pixels. On the synthetic repro — a 40x90 px subject moving in the last 10 of 65 frames — v0.7.15 kept 1 of 10 action frames and v0.7.16 kept all 10.

Measured Cost

The README publishes numbers rather than claims. On a 3-minute 640x360 video on a Mac mini M4, frames and dedup only:

Token estimates use Anthropic's (width x height) / 750, about 307 tokens per frame at that resolution. Fewer frames is the whole point: context is the budget.

Surfaces

The tool ships in more forms than a CLI. crv-web opens a local page in Traditional Chinese, Simplified Chinese or English for people who do not want a terminal. An MCP server (crv-mcp) exposes watch_video and get_frames to Claude Desktop, Claude Code and Cursor, caching analyses under ~/.cache/crv-mcp so follow-ups are instant. It also installs as an agent skill via npx skills add, or as a Claude Code plugin through its own marketplace. There is a Python API (from claude_real_video import process).

Mode flags cover the cases where pure scene detection underperforms: --adaptive compares against a rolling neighbourhood for slow morphs and gradual pans; --text-anchors forces frames at subtitle-cue timestamps for lecture slides and screen recordings; --speakers runs a 45 MB local diarization model to label transcript lines. --why "find the pricing strategy" writes intent into the manifest so the analysis has a lens, and --kb files the result as a dated note in your own notes folder.

Privacy Posture

Processing runs on the local machine — fetch, extract, dedup and transcribe all happen there, and the source video is never uploaded. The README is careful to state the actual boundary: whatever frames or transcript text you subsequently paste into a cloud LLM does go to that provider. That is the honest framing, and it is more useful than an unqualified "private" badge.

Caveats

There is a paid tier, and the README sells it. The free MIT tool lets a model see the video; crv Pro, a $29 one-time add-on sold through Capafy and Lemon Squeezy, is pitched as letting it understand the video — cut rhythm, camera moves, gestures, expressions, voice pitch, sound events. The open-source core is genuinely complete and usable, but the README is also a storefront, and the split between free and paid capability is a maintainer decision that can move.

Setup is not pip-only. ffmpeg/ffprobe must be installed separately. Transcription requires the [whisper] extra — without it there is no speech-to-text, though videos shipping their own subtitles still produce a transcript. Diarization, MCP and faster-whisper are three more extras.

Transcript quality is capped by the model you pick. The default is Whisper base. Sharper output means --whisper-model turbo, which is a one-time 1.6 GB download and roughly 6 GB of memory.

Tuning is real work. --scene, --dedup-threshold, --dedup-window, --fps-floor and --max-frames all interact. --report exists precisely because you will want to see every keep/drop decision — which is a good tool, and also an admission that defaults will not fit every video.

It is five weeks old. One primary maintainer, 5 contributors, 16 patch releases in that span. The velocity is encouraging and the v0.7.16 blind-spot fix shows the maintainer responds to real failure reports, but the interface is clearly still moving.

Verdict

The insight worth taking from crv is that "multimodal" for most LLM workflows is still a preprocessing problem, not a model problem. Uniform frame sampling wastes context on redundancy and misses the moments that carry meaning; scene-aware selection with windowed dedup spends the same token budget on more information.

Anyone routinely asking an AI agent about screen recordings, lectures, demos or reels should try it — the MCP server is the lowest-friction entry point, since it turns "watch this video" into a tool call. Teams should read the free-versus-Pro boundary before building a workflow on top of it.