inclusionAI released Ling-3.0-tiny-UBTree speculative-decoding draft model on Hugging Face
inclusionAI released Ling-3.0-tiny-UBTree, a 183.1M-parameter speculative-decoding draft model for Ling-3.0-tiny, on Hugging Face under MIT license. It reports up to 3.518× end-to-end speedup over target-only autoregressive decoding on MATH-500. SGLang integration is not yet publicly available.
China context
- Original name
- 蚂蚁集团
- Outside China
- Open weights · huggingface.co
- Claims
- Company-reported; not yet independently evaluated
- For builders
- Builders outside China can download the MIT-licensed weights from Hugging Face and experiment with the draft model, but cannot yet use it in production serving because the SGLang integration is not publicly available.
- For investors
- The release demonstrates Ant Group's continued investment in inference efficiency for open-weights models, but the lack of a public serving integration means the commercial impact is not yet measurable.
inclusionAI (Ant Group) released Ling-3.0-tiny-UBTree on Hugging Face, a speculative-decoding draft model for the Ling-3.0-tiny target model. The draft uses a five-layer DFlash backbone and a learned bigram selector to propose multiple candidate continuations for target-model verification. The model card reports benchmark results with block size 16, TP1, concurrency 1, BF16, and greedy no-thinking decoding, showing end-to-end speedups ranging from 2.794× on LiveCodeBench to 3.518× on MATH-500. The weights are released under the MIT License, but the SGLang serving integration is not yet publicly available.
The draft model has 183,127,326 parameters, hidden size 1,536, 16 attention heads and 16 KV heads, and uses BF16 weights. It shares token embeddings and the language-model head with the target model, but those shared weights are not included in the checkpoint. The UBTree method pairs a unigram proposer with a bigram selector, constructs a verification tree with a default budget of 64 non-root nodes plus one anchor, and uses tree-native training with renormalized KL supervision on high-temperature target traces. Reported acceptance lengths range from 7.46 on LiveCodeBench to 10.04 on HumanEval.
Developers using Ling-3.0-tiny can potentially reduce inference latency by up to 3.518× on reasoning-heavy tasks by adopting this draft model, but they must wait for the SGLang integration to be released before deployment. The MIT license allows commercial use and modification, which lowers the barrier for enterprises to integrate speculative decoding into their serving stacks.
For organizations serving Ling-3.0-tiny at scale, the reported 2.794× to 3.518× end-to-end speedup could reduce GPU serving costs proportionally once the SGLang integration is available. The MIT license permits commercial use without royalty, but the current lack of public serving integration means the value is not yet realizable.
The next verifiable signal is the release of the SGLang integration for UBTree with Ling-3.0-tiny, including installation instructions and serving recipes. Independent benchmarks on standard hardware configurations would confirm whether the reported speedups hold outside the specified TP1, concurrency 1, BF16, greedy decoding setup.