PaddlePaddle releases HALOps, an open-source high-performance operator library for LLM inference on NVIDIA Blackwell
PaddlePaddle (Baidu) released HALOps, an open-source high-performance operator library for LLM inference on NVIDIA Blackwell (SM100) GPUs. It covers attention, MoE, and MLA paths, and is available under Apache License 2.0.
China context
- Original name
- HALOps
- Outside China
- Open weights · github.com
- Claims
- Company-reported; not yet independently evaluated
- For builders
- Developers outside China can access the HALOps source code and prebuilt wheels for Linux x86_64 and aarch64, enabling integration into their own inference stacks on NVIDIA Blackwell hardware.
- For investors
- The release signals Baidu's continued investment in high-performance inference infrastructure, which may influence the competitive landscape for AI hardware and cloud services.
HALOps (High-performance Accelerated LLM Operators) is a high-performance operator library for large language model inference, developed by the ERNIE team at Baidu. It is designed for NVIDIA Blackwell (SM100) GPUs and built on CUTLASS/CuTe, using tcgen05, TMEM, and TMA. The library includes GQA attention, multi-precision GEMM (FP8/MXFP8, FP4/NVFP4/MXFP4), and MegaKernel sparse MLA. Prebuilt wheels are available for Python 3.12 with CUDA 13.2 on Linux x86_64 and aarch64. The repository is released under Apache License 2.0.
HALOps targets NVIDIA Blackwell SM100 GPUs and requires CUDA 13.0+, driver R580+, Python 3.10+, and PyTorch 2.13+. It uses PDL, multi-stream overlap, operator fusion, and MegaKernels to reduce kernel bubbles. Performance claims include MXFP8 dense GEMM outperforming DeepGEMM and FlashInfer in some cases, and MegaMLA outperforming multi-stream + PDL at small concurrency. These benchmarks are company-reported and not independently verified.
Developers building LLM inference on NVIDIA Blackwell GPUs gain a new open-source operator library that may improve throughput and latency for attention, MoE, and MLA workloads. Baidu's release of HALOps under Apache 2.0 allows competitors and researchers to inspect and reuse the code, potentially raising the performance bar for inference kernels.
HALOps is open-source and free to use under Apache 2.0, potentially reducing development costs for teams optimizing LLM inference on NVIDIA Blackwell. It may also serve as a reference for hardware vendors and cloud providers looking to improve inference efficiency.
The roadmap includes adding W4A8, BF16, and other GEMM types, higher-performance MegaKernels, and kernels tuned for small workloads. Verification of the claimed performance advantages against DeepGEMM and FlashInfer would require independent benchmarking on SM100 hardware.