Tencent open-sources Hy4-preview, a 770B-parameter MoE flagship model
Tencent released Hy4-preview, a 770B-parameter Mixture-of-Experts model with 49B activated parameters and 1M context length, under Apache-2.0 on Hugging Face, ModelScope, GitCode, and CNB. The release includes FP8 quantized weights and supports vLLM and SGLang deployment.
China context
- Original name
- 腾讯混元 Hy4-preview
- Outside China
- Open weights · huggingface.co
- Claims
- Company-reported; not yet independently evaluated
- For builders
- Builders can self-host Hy4-preview using vLLM or SGLang and fine-tune it under Apache-2.0, avoiding API rate limits and data-sharing requirements. The 1M context length and MoE efficiency make it suitable for long-document processing and agentic workflows.
- For investors
- Tencent's open-weights release intensifies competition in the frontier open-source model market, which may compress margins for API-only providers. Investors should monitor adoption metrics on Hugging Face and downstream enterprise deployments.
Tencent's Hy Team open-sourced Hy4-preview, a new-generation MoE flagship model with 770B total parameters, 49B activated per token, and a 1M context window. The model uses Gated DeepSeek Sparse Attention with IndexCache and identity Hyper-Connections. It is available on Hugging Face, ModelScope, GitCode, and CNB, with FP8 quantized weights and deployment recipes for vLLM and SGLang.
Hy4-preview uses a 78-layer backbone with 256 routed experts and 1 shared expert per MoE layer, activating top-8 routed experts per token. It includes a native MTP layer (10B total, 0.7B activated) for speculative decoding. Attention uses Gated DSA with 64 heads, query compression dimension 2048, key-value compression 512, and indexer top-k 2048. Residual streams are 4, and vocabulary size is 120,832.
Developers outside China gain immediate access to a frontier-scale open-weights model with Apache-2.0 licensing, enabling self-hosted deployment and fine-tuning without API costs or data-sharing constraints. This pressures other open-weights providers to match parameter scale and context length while offering permissive licenses.
Enterprises can deploy Hy4-preview on their own infrastructure, reducing per-token costs compared to API-based frontier models and avoiding data egress. The model's focus on software engineering, office analysis, game development, and scientific research targets high-value productivity use cases.
Tencent states it will continue iterating on Hy4-preview, addressing known issues such as over-reasoning and over-verification. The model is co-designed with Tencent products CodeBuddy and WorkBuddy, suggesting future integration into enterprise workflows. Independent benchmarks and community evaluations will be key signals for adoption.