Reportedly: Moore Threads open-sources Megatron-MUSA-Titan for trillion-parameter MoE training on 10,000-GPU clusters
Reported by IT Home and Pandaily · not yet confirmed by the company or a second independent outlet. We update this page when it is.
Moore Threads open-sourced Megatron-MUSA-Titan, its training stack for trillion-parameter MoE pretraining on 10,000-GPU clusters, IT Home reported. The project supports blockwise FP8 training and works with slime and miles reinforcement learning frameworks. The code is available on GitHub.
China context
- Original name
- 摩尔线程
- Outside China
- Open weights · ithome.com
- Claims
- Company-reported; not yet independently evaluated
- For builders
- Developers can access the code on GitHub and potentially adapt it for MUSA-based systems, but compatibility with non-MUSA hardware is not stated.
- For investors
- The open-sourcing may signal Moore Threads' push to build a software ecosystem around its GPUs, but revenue impact is unclear without adoption data.
Translated from Chinese. Quotes and facts link to the original sources.
Moore Threads announced the open-source release of Megatron-MUSA-Titan, a large model training infrastructure project based on its full-function GPUs. The project targets trillion-parameter Mixture-of-Experts (MoE) model training on 10,000-GPU clusters, covering both pretraining and reinforcement learning. It has been used in Moore Threads' own 10,000-GPU cluster for trillion-parameter MoE pretraining, achieving high Model FLOPs Utilization (MFU) and long-duration stable training. The stack supports blockwise FP8 training, integrates with DeepEP and ACE asynchronous communication engines to accelerate token dispatch and result merging in MoE, and enables overlap of expert parallelism and computation. It also supports collaboration with slime and miles reinforcement learning frameworks, combining Megatron training with SGLang sample generation.
Megatron-MUSA-Titan adapts NVIDIA's Megatron framework to Moore Threads' MUSA platform, enabling trillion-parameter MoE training on domestic GPUs. Key techniques include blockwise FP8 low-precision training, asynchronous communication via DeepEP and ACE for MoE token dispatch, and expert parallelism overlapping with computation to improve end-to-end efficiency. The reported high MFU and long-stability training on a 10,000-GPU cluster are company-reported and not independently verified.
Developers outside China can now inspect and potentially adapt Moore Threads' training stack for large-scale MoE models, reducing dependence on NVIDIA's CUDA ecosystem for trillion-parameter training. This move may lower the barrier for Chinese GPU users to train frontier-scale models, but the actual performance and compatibility with non-MUSA hardware remain unverified.
For enterprises and research institutions using Moore Threads GPUs, this open-source stack provides a reference implementation for large-scale MoE training, potentially reducing engineering effort. However, the business value depends on the availability and cost of Moore Threads' 10,000-GPU clusters and the maturity of the MUSA software ecosystem.
A verifiable next signal is whether third-party developers adopt Megatron-MUSA-Titan and report independent benchmark results on Moore Threads GPUs. Another signal is whether Moore Threads releases pretrained model weights or detailed scaling data from its 10,000-GPU cluster.