OpenBMB releases llama.cpp-omni, an open-source C++ engine for full-duplex omni-modal streaming
OpenBMB released llama.cpp-omni, a C++ inference engine supporting full-duplex omni-modal streaming video calls, built on llama.cpp and MiniCPM-o 4.5. It supports Windows, Linux, and macOS, with GGUF quantization and low memory footprint. The source release was published on 2026-10-13.
China context
- Original name
- 面壁智能
- Outside China
- Open weights · github.com
- Claims
- Company-reported; not yet independently evaluated
- For builders
- Developers outside China can integrate llama.cpp-omni into existing llama.cpp toolchains and deploy omni-modal models on consumer hardware, enabling real-time voice and video applications without cloud dependencies.
- For investors
- The release of an open-source, efficient omni-modal inference engine by ModelBest may accelerate adoption of on-device multimodal AI, potentially impacting the market for proprietary omni-modal API services.
OpenBMB (ModelBest) released llama.cpp-omni, a high-performance C++ inference engine for omni-modal models, built on llama.cpp. It is the first open-source C++ framework supporting full-duplex, omni-modal streaming video calls. The engine is compatible with llama.cpp interfaces, supports GGUF quantization, and runs on Windows, Linux, and macOS. It is designed for the MiniCPM-o 4.5 model, a 9B-parameter on-device omni-modal LLM developed by ModelBest and Tsinghua University. The engine splits the model into GGUF modules: VPM (vision encoder), APM (audio encoder), LLM (Qwen3-8B), TTS, and Token2Wav vocoder. It implements full-duplex streaming with time-division multiplexing and proactive interaction at 1Hz. Benchmarks show TTFT under 550ms on RTX 4090 and under 650ms on Apple M4 Max.
llama.cpp-omni modularizes MiniCPM-o 4.5 into separate GGUF components (VPM, APM, LLM, TTS, Token2Wav) and uses time-division multiplexing within the LLM backbone to synchronize input and output streams at millisecond level. It supports streaming encoders with 1-second audio chunks and frame-by-frame video, and a sliding-window Token2Wav (28 tokens in, 25 tokens stride) for real-time audio synthesis. Quantization options include F16, Q8_0, and Q4_K_M, with full omni VRAM estimates from ~20GB (F16) to ~9GB (Q4_K_M) on NVIDIA GPUs.
Developers outside China can now run a full-duplex omni-modal model locally on consumer hardware, reducing reliance on cloud APIs for real-time voice and video applications. This lowers the barrier for building interactive AI agents and may pressure proprietary omni-modal API providers to match on-device latency and cost.
For developers, llama.cpp-omni offers a free, cross-platform way to deploy omni-modal models with low latency and memory usage, potentially reducing infrastructure costs. For enterprises, it enables on-premise or edge deployment of voice and video AI without per-token API fees, though support and maintenance depend on the open-source community.
Watch for community adoption of llama.cpp-omni in open-source projects and whether ModelBest releases updated MiniCPM-o models optimized for this engine. A key signal is the number of GitHub stars and forks in the first month, and any third-party benchmarks comparing its latency to cloud-based omni-modal APIs.