Alibaba PAI open-sourced a ControlNet-Union for MiniMax-H3 video generation
alibaba-pai released MiniMax-H3-Fun-Controlnet-Union on Hugging Face, a ControlNet-Union for MiniMax-H3 supporting Canny, Depth, HED, MLSD, Pose, and video inpainting. The checkpoint is guidance-distilled and runs with guidance scale 1.0. A newer version, MiniMax-H3-Fun-Controlnet-Union-2.0, is recommended.
China context
- Outside China
- Open weights · huggingface.co
- Claims
- Company-reported; not yet independently evaluated
- For builders
- Builders can integrate this ControlNet-Union into video generation pipelines, but must handle the high memory requirements and comply with the MiniMax H3 Community License Agreement.
- For investors
- The release of a ControlNet-Union for MiniMax-H3 by Alibaba PAI indicates ongoing investment in open-weights video generation tooling, but the legacy status and recommendation of a 2.0 version suggest rapid iteration.
alibaba-pai released MiniMax-H3-Fun-Controlnet-Union on Hugging Face. This is a ControlNet-Union for MiniMax-H3, trained with the VideoX-Fun pipeline. A single checkpoint conditions the MiniMax-H3 video generator on Canny, Depth, HED, MLSD or Pose control videos, and also runs video inpainting. The model card notes that this is a legacy version and recommends MiniMax-H3-Fun-Controlnet-Union-2.0 for new usage.
The control branch attaches to 5 of the 50 transformer blocks (layers 0, 10, 20, 30, 40) with zero-gated projections. The checkpoint is guidance-distilled, requiring guidance scale = 1.0. The control input is widened to control in dim = 49 for inpainting. Memory requirements are high: the transformer (~62 GB) plus Qwen3-VL text encoder (~62 GB) do not fit one 80 GB GPU fully loaded; model group offload or model cpu offload with qfloat8 is needed.
Developers outside China who want controllable video generation with MiniMax-H3 can now use this open-weights ControlNet-Union, but they must comply with the MiniMax H3 Community License Agreement, including territorial restrictions and Acceptable Use Policy.
This release provides a single checkpoint for multiple control modalities, reducing the need to switch between models for different control types. However, the high memory requirements may limit accessibility for users with smaller GPU setups.
The model card explicitly recommends MiniMax-H3-Fun-Controlnet-Union-2.0 for new usage, indicating that this legacy version may not receive further updates. Verification question: whether the 2.0 version is already available and what improvements it offers.