OpenBMB released MiniCPM-V-4.7-35B-A3B-DSpark, a speculative decoding draft model, on Hugging Face
OpenBMB released MiniCPM-V-4.7-35B-A3B-DSpark on Hugging Face, a 5-layer draft model trained to accelerate MiniCPM-V-4.7-35B-A3B. It is Apache-2.0 licensed and achieves acceptance lengths up to 7.387 on code tasks.
China context
- Original name
- 面壁智能
- Outside China
- Open weights · huggingface.co
- Claims
- Company-reported; not yet independently evaluated
- For builders
- Developers can integrate this draft model with MiniCPM-V-4.7-35B-A3B to reduce inference latency using speculative decoding, with Apache-2.0 licensing permitting commercial use.
- For investors
- The release of an open-weights acceleration component for a large vision-language model may reduce deployment costs for adopters, potentially increasing the attractiveness of ModelBest's ecosystem.
MiniCPM-V-4.7-35B-A3B-DSpark is a speculative decoding draft checkpoint designed for exact pairing with MiniCPM-V-4.7-35B-A3B and its tokenizer. The model has 5 draft layers, 389,223,169 parameters, and generates 15 draft tokens per forward pass. It was trained on 2M sequences for 6 epochs using responses generated by the target model from a mixture of vision and text prompts. Evaluation shows acceptance lengths ranging from 3.501 (general text) to 7.387 (code). The model is available on Hugging Face under Apache-2.0 license, with a llama.cpp GGUF version also provided.
The draft model uses a target hidden-state layer set of [3, 11, 19, 27, 35] and is trained with a combined objective of cross-entropy, L1, and confidence loss. It achieves an acceptance length of 5.098 on MMMU and 6.126 on MathVista, indicating effective speculative decoding for multimodal tasks. The model is designed for use with vLLM.
Developers using MiniCPM-V-4.7-35B-A3B can reduce inference latency by pairing it with this draft model, potentially lowering serving costs for vision-language applications. The Apache-2.0 license allows commercial use without additional fees.
The release provides a ready-to-use acceleration component for MiniCPM-V-4.7-35B-A3B, which may improve cost-efficiency for developers deploying the model in production.
Adoption can be tracked by monitoring downloads and community feedback on the Hugging Face model page, as well as any integration into popular serving frameworks beyond vLLM.