Reportedly: Baidu open-sources ERNIE 5.5 Flash Base, a 280B-parameter MoE model
Reported by IT Home and Leiphone · not yet confirmed by the company or a second independent outlet. We update this page when it is.
Baidu released ERNIE-5.5-Flash-Base, a sparse MoE model with 280B total parameters and 14B active per token, pretrained on 33.5T tokens, IT Home reported. Weights are available on Hugging Face. The model supports 128K training context and 1M inference context.
China context
- Original name
- 百度文心 ERNIE 5.5 Flash Base
- Outside China
- Open weights · leiphone.com
- Claims
- Company-reported; not yet independently evaluated
- For builders
- Developers can download the base model weights from Hugging Face and use them for domain adaptation, instruction tuning, or RL research without pretraining from scratch.
- For investors
- Baidu's open-weight release intensifies competition in the open model space, potentially pressuring other labs to release comparable models or differentiate on post-training and tooling.
Translated from Chinese. Quotes and facts link to the original sources.
Baidu has open-sourced ERNIE 5.5 Flash Base, a sparse mixture-of-experts language model with 280 billion total parameters and 14 billion active parameters per token. The model was pretrained from scratch on 33.5 trillion tokens using the Muon optimizer and FP8 mixed precision. It supports a maximum training context length of 128K tokens, and fine-tuning and inference support up to 1M tokens. The released weights are the base model without mid-training or post-training, intended as a starting point for domain adaptation, instruction tuning, and reinforcement learning research. Baidu states that a post-trained ERNIE 5.5 Flash will be released later. The model reportedly achieves state-of-the-art results among models of similar size on benchmarks covering knowledge, reasoning, math, code, multilingual, and long-context tasks, surpassing DeepSeek-V4-Flash-Base, Hy3-preview-Base, and MiMo-V2.5-Base. Architectural innovations include Latent-Gated Sparse Attention, Multi-Router Latent MoE, and ECHO training objective. The model is available on Hugging Face at https://huggingface.co/baidu/ERNIE-5.5-Flash-Base.
ERNIE 5.5 Flash Base uses a sparse MoE architecture with 280B total parameters and 14B active per token. It introduces Latent-Gated Sparse Attention, combining hybrid sparse attention, low-rank gated attention, and virtual head attention to reduce long-context resource overhead. Multi-Router Latent MoE uses multiple independent routers for expert selection. The ECHO training objective reuses the backbone's last layer for multi-token prediction without an additional MTP module, improving reasoning and code ability. Training used a distributed Muon optimizer and FlashMask-4 attention operator for large head dimensions.
Developers outside China gain immediate access to a 280B-parameter open-weight base model, enabling domain adaptation and instruction tuning without pretraining costs. Competitors must now match Baidu's efficiency claims on long-context and MoE routing, or risk losing open-weight mindshare.
Open-weight release lowers the barrier for enterprises and researchers to build on a large-scale Chinese model, potentially reducing costs for domain-specific fine-tuning. The model's efficiency focus may translate to lower inference costs for long-context applications.
The post-trained ERNIE 5.5 Flash is expected to be released later, which will provide a more directly usable model for applications. Independent evaluations of the base model's benchmark claims and the efficiency of its attention and routing mechanisms will be key signals to watch.