VibeASR.cpp is the official inference runtime for Microsoft's VibeVoice-ASR-BitNet, and its whole reason for existing is a single question: can a modern LLM-based speech recognizer run in real time on an ordinary CPU? The repository has drawn 143 stars and 18 forks since its 23 July 2026 release, with the weights pulling 16,586 downloads and 182 likes on Hugging Face. The answer it offers is heterogeneous quantization — I8_S for the VAE tokenizer and I2_S (ternary BitNet) for the language-model decoder — rather than the uniform int4/int8 treatment most runtimes apply.

The size story is the clearest part. Microsoft first swaps the original Qwen2.5-7B language model for Qwen2.5-1.5B, then quantizes each half differently: the VAE tokenizer drops from 1.31 GB to 0.65 GB (2.0x) and the LM decoder from 3.32 GB to 0.92 GB (3.6x), taking the total from 4.62 GB to 1.58 GB — a 2.9x compression. The reported accuracy cost of the 7B-to-1.5B substitution is a 1-4% absolute WER increase, which is the kind of trade the README states plainly rather than burying.

Speed comes from custom SIMD kernels and operator fusion inside ggml, and the project reports 1.6-2.3x faster inference than Whisper.cpp at comparable model sizes. The published tables are more specific: on an AMD EPYC 7V13 the real-time factor is 1.98 single-threaded but 0.77 at three threads and 0.42 at eight, with the speedup over Whisper.cpp measured at 2.28x at one thread narrowing to 1.55x at eight. An Apple M4 hits RTF 0.68 at two threads, and an Intel Core i7-13700 reaches 0.97 at two threads. Anything under 1.0 is faster than real time, so the practical claim is that a two-to-four-thread consumer CPU is enough.

On accuracy the picture is mixed and worth reading carefully. The BitNet build posts 8.25 WER on MLC-EN against Parakeet's 8.40, Whisper's 13.57, SenseVoice's 12.39 and FunASR's 11.36, and it leads on the harder meeting sets — 21.36 on AMI-ihm and 25.87 on AMI-sdm, versus 27.07 and 36.92 for Whisper. It loses on clean read speech: 2.41 on LibriSpeech clean where Parakeet manages 1.49, and 5.21 on Fleurs-en where Whisper reaches 3.99. Multilingual coverage spans MLC French, Italian, Korean, Portuguese and Vietnamese, though WER there runs from 11.15 (Korean) to 24.87 (Portuguese). The README also carries an explicit caveat that these are standard-accent corpora and that accented or dialectal speech may degrade further.

Setup is deliberately small: Python 3.9+, CMake 3.14+, a C++11 GCC or Clang toolchain, and about 2 GB of disk for code plus the pre-quantized GGUF files (vibeasr-vae-encoder-i8_s.gguf and vibeasr-lm-i2_s-embed-q6_k.gguf). A setup_env.py one-command path, an asr_infer CLI and a Gradio web demo are all included. The one real friction point is Windows: MSVC is rejected outright by the build and MinGW-w64 GCC or Clang is required, which is a meaningful obstacle for the CPU-first Windows audience this project would otherwise serve well. Licensing is MIT and a technical report is published at arXiv:2607.21075.