**claude-real-video** (`crv` on PyPI) is a local preprocessing tool that turns a video into something a text-and-image LLM can actually read: the frames that matter, deduplicated, plus a timestamped transcript and a manifest. Point it at a YouTube link or a local file, and it writes `crv-out/frames/*.jpg`, `frames.json`, `transcript.txt/.json` and `MANIFEST.txt` — then you paste that into Claude, ChatGPT or Gemini and ask questions. The repository was created on **30 June 2026** and has reached roughly **1,951 stars and 163 forks**, with the latest release (**v0.7.16**) on 21 July and commits through 3 August. It is MIT-licensed with 5 contributors, and reached the Hacker News front page. ## The Problem It Targets Most "let an LLM watch a video" paths do something weaker than they appear to. Paste a YouTube link into ChatGPT and it reads the transcript, not the picture. Claude does not accept a video file at all. Gemini can read video natively, but it uploads the file and samples frames at a **fixed interval** — 1 fps by default — which over-samples a static screencast and under-samples a fast-cut reel. crv replaces the fixed quota with **scene-change detection plus a density floor**, then removes near-duplicates. On a 58-second clip, 1 fps sampling yields 58 frames; crv keeps the **26 that actually differ**. ## Pipeline | Stage | Implementation | |---|---| | Fetch | `yt-dlp` for URLs (optional cookies) or a local file copy | | Extract | One chronological `ffmpeg select` pass — every scene change plus at least one frame per `--fps-floor` seconds | | Dedup | Three channels compared against a sliding window of the last `--dedup-window` kept frames | | Transcribe | Whisper, or faster-whisper with Silero VAD when the `[fast]` extra is installed | | Output | Frames, `frames.json`, transcript, `MANIFEST.txt`, optional `viewer.html` and contact-sheet grids | The dedup design is where the engineering sits. A **global channel** measures real pixel difference on downscaled RGB rather than a perceptual hash — the README notes hashes go blind on flat colours and equal-luma hue changes. A **settled-local channel** catches what the global one averages away: thin pen strokes, caption swaps and small UI updates, gated so it only fires on a *settled* new state rather than motion mid-flight, with 1px shift tolerance so grain and jitter do not trigger it. Because the window spans the last N kept frames rather than only the previous one, an A-B-A cutaway does not re-send a shot the model has already seen. Version **0.7.16** added a third "action" channel after a structural blind spot surfaced in a user's 2,181-video batch run: a percentage comparator can never detect a subject covering under 1% of the frame, because it cannot change 8% of the pixels. On the synthetic repro — a 40x90 px subject moving in the last 10 of 65 frames — v0.7.15 kept 1 of 10 action frames and v0.7.16 kept all 10. ## Measured Cost The README publishes numbers rather than claims. On a 3-minute 640x360 video on a Mac mini M4, frames and dedup only: | Mode | Frames kept | Wall time | Est. image tokens | |---|---|---|---| | default | 170 | 23.5 s | ~52k | | `--max-frames 80` | 80 | 23.4 s | ~25k | | `--adaptive` | 270 | 36.8 s | ~83k | Token estimates use Anthropic's `(width x height) / 750`, about 307 tokens per frame at that resolution. Fewer frames is the whole point: context is the budget. ## Surfaces The tool ships in more forms than a CLI. `crv-web` opens a local page in Traditional Chinese, Simplified Chinese or English for people who do not want a terminal. An **MCP server** (`crv-mcp`) exposes `watch_video` and `get_frames` to Claude Desktop, Claude Code and Cursor, caching analyses under `~/.cache/crv-mcp` so follow-ups are instant. It also installs as an agent skill via `npx skills add`, or as a Claude Code plugin through its own marketplace. There is a Python API (`from claude_real_video import process`). Mode flags cover the cases where pure scene detection underperforms: `--adaptive` compares against a rolling neighbourhood for slow morphs and gradual pans; `--text-anchors` forces frames at subtitle-cue timestamps for lecture slides and screen recordings; `--speakers` runs a 45 MB local diarization model to label transcript lines. `--why "find the pricing strategy"` writes intent into the manifest so the analysis has a lens, and `--kb` files the result as a dated note in your own notes folder. ## Privacy Posture Processing runs on the local machine — fetch, extract, dedup and transcribe all happen there, and the source video is never uploaded. The README is careful to state the actual boundary: whatever frames or transcript text you subsequently paste into a cloud LLM does go to that provider. That is the honest framing, and it is more useful than an unqualified "private" badge. ## Caveats **There is a paid tier, and the README sells it.** The free MIT tool lets a model *see* the video; **crv Pro**, a $29 one-time add-on sold through Capafy and Lemon Squeezy, is pitched as letting it *understand* the video — cut rhythm, camera moves, gestures, expressions, voice pitch, sound events. The open-source core is genuinely complete and usable, but the README is also a storefront, and the split between free and paid capability is a maintainer decision that can move. **Setup is not pip-only.** `ffmpeg`/`ffprobe` must be installed separately. Transcription requires the `[whisper]` extra — without it there is no speech-to-text, though videos shipping their own subtitles still produce a transcript. Diarization, MCP and faster-whisper are three more extras. **Transcript quality is capped by the model you pick.** The default is Whisper `base`. Sharper output means `--whisper-model turbo`, which is a one-time 1.6 GB download and roughly 6 GB of memory. **Tuning is real work.** `--scene`, `--dedup-threshold`, `--dedup-window`, `--fps-floor` and `--max-frames` all interact. `--report` exists precisely because you will want to see every keep/drop decision — which is a good tool, and also an admission that defaults will not fit every video. **It is five weeks old.** One primary maintainer, 5 contributors, 16 patch releases in that span. The velocity is encouraging and the v0.7.16 blind-spot fix shows the maintainer responds to real failure reports, but the interface is clearly still moving. ## Verdict The insight worth taking from crv is that "multimodal" for most LLM workflows is still a preprocessing problem, not a model problem. Uniform frame sampling wastes context on redundancy and misses the moments that carry meaning; scene-aware selection with windowed dedup spends the same token budget on more information. Anyone routinely asking an AI agent about screen recordings, lectures, demos or reels should try it — the MCP server is the lowest-friction entry point, since it turns "watch this video" into a tool call. Teams should read the free-versus-Pro boundary before building a workflow on top of it.