Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
MiniCPM-V is OpenBMB's Apache-2.0 pocket-sized multimodal LLM series that has crossed 25,500 GitHub stars by making image and video understanding viable on phones rather than only datacenter GPUs. The May 11, 2026 release of MiniCPM-V 4.6 introduced mixed 4x/16x visual token compression that cuts visual encoding cost by more than 50% while keeping accuracy competitive with much larger models, and the project ships first-party iOS, Android, and HarmonyOS deployment code rather than leaving edge integration as an exercise for the reader. ## What MiniCPM-V Is For The project targets a specific gap: developers who need GPT-4o-class image and video understanding but cannot ship a 70B-parameter cloud call into a mobile app. A 1.3B-parameter MiniCPM-V 4.6 checkpoint handles single-image, multi-image, and short-video queries with roughly 1.5x the token throughput of competing 0.8B models, which is the difference between a usable on-device assistant and a slideshow. The companion MiniCPM-o 4.5 release, also from 2026, pushes this to a 9B full-duplex omnimodal model that approaches Gemini 2.5 Flash on vision and speech benchmarks. ## Mixed 4x/16x Visual Token Compression The headline architectural change in 4.6 is intra-ViT early compression with a mixed 4x/16x token reduction strategy. Visual tokens are the dominant cost in multimodal inference, and prior open MLLMs left this fixed at the encoder. MiniCPM-V 4.6 lets the model spend a 4x budget on regions that matter for the prompt and 16x compression on regions that do not, which is what produces the 50%+ visual encoding cost reduction without the accuracy collapse that uniform aggressive compression usually causes. ## First-Party Edge Deployment Path Most open multimodal projects stop at a Hugging Face checkpoint and a vLLM serving example. MiniCPM-V ships actual iOS, Android, and HarmonyOS adaptation code, plus quantized variants in GGUF, BNB, AWQ, and GPTQ formats, plus integrations with vLLM, SGLang, llama.cpp, and Ollama. For a team that wants to put vision-language inference inside a mobile app this week, the path from clone to working build is materially shorter than with InternVL or LLaVA-class projects. ## Omnimodal Streaming with MiniCPM-o The MiniCPM-o 4.5 variant, released February 3, 2026, adds full-duplex audio plus video streaming in a single end-to-end model. Most open omnimodal stacks today are an LLM with bolted-on Whisper and TTS, and the seams show in latency. MiniCPM-o handles speech and video natively, which is closer to what GPT-4o demonstrated as a product target. Combined with the 4.6 vision lineage, this gives the project a unified family that covers both lightweight on-device and full real-time conversational deployments. ## Free Public API In May 2026 OpenBMB launched a free public API for both the 4.6 vision and 4.5 omnimodal models. For prototyping that hits the same model behavior the on-device build will eventually serve, this removes the usual gap between cloud-API exploration and edge implementation. Teams can validate prompts and integrations against the API, then swap the same model in on-device once the product spec is locked. ## Limitations Document understanding above ~128K tokens and very long videos still favor larger MLLMs like InternVL3.5 and Qwen3-VL — MiniCPM-V's compression strategy and parameter budget pay back fastest on short-to-medium-length inputs. Reasoning-heavy STEM and math benchmarks also remain a weakness for the lightweight checkpoints, which is the trade-off of optimizing for edge throughput. Finally, the model weights are released under a license that requires registration for commercial use even though the code is Apache-2.0, so commercial integrators should review weight terms separately. Within those constraints, MiniCPM-V is the most credible open answer in 2026 for teams that need pocket-deployable multimodal understanding rather than another datacenter-class checkpoint.
hacksider
Real-time AI face swap and one-click video deepfake with only a single image
harry0703
AI-powered short video generator that automates scripting, footage sourcing, subtitles, and composition — supporting 10+ LLM providers and batch production.