Qwen released Qwen3.8-Flash-Next on Hugging Face
Qwen (Alibaba) released Qwen3.8-Flash-Next on Hugging Face. It is an image-text-to-text model with 125B total parameters (6B activated), plus 51B n-gram embedding and 4B MTP. Context length is 262,144 natively, extensible to 1,000,000 tokens. License: other. Library: transformers.
China context
- Original name
- 通义千问
- Outside China
- Open weights · huggingface.co
- Claims
- Company-reported; not yet independently evaluated
- For builders
- Developers outside China can download the weights from Hugging Face and test the new architecture for long-context and agentic applications.
- For investors
- The open-weight release allows investors to assess the technical direction of Qwen4 and gauge developer interest before commercial API launch.
Qwen3.8-Flash-Next is an experimental preview of the architecture that will underpin Qwen4. It introduces Hybrid Attention with Qwen Sparse Attention (QSA), Gated Residual, N-gram Embedding, and a tailored training recipe using Muon and AdamW optimizers. The model is available on Hugging Face with weights and configuration files compatible with Transformers, vLLM, SGLang, and TokenSpeed.
QSA operates at the micro-block level rather than selecting individual tokens, cutting long-context latency. Gated Residual uses an element-wise data-dependent read gate and per-branch scalar write gate. N-gram Embedding indexes with short n-grams for efficient parameter scaling. The training recipe eliminates batch-size warmups and starts directly at the target batch size.
This release signals Qwen's architectural direction for Qwen4, emphasizing efficiency for agentic workloads and long-context processing. The open-weight release allows developers to test the architecture before the official Qwen3.8-Flash version, which will include production features like 1M context by default and built-in tools.
For builders, the open weights enable experimentation with a new architecture optimized for long-context and agentic tasks. For investors, the release indicates Alibaba's continued investment in frontier model efficiency and its strategy to seed the ecosystem before commercial API offerings.
Observable next signals include the release of the official Qwen3.8-Flash with production features, further technical reports or blog posts detailing the architecture, and community feedback on the experimental preview. Adoption of the architecture in downstream applications can be tracked via Hugging Face downloads and derivative models.