Kimi k1.5: Scaling Reinforcement Learning with LLMs: Reinforcement Learning Drives Multimodal Reasoning, Performance Comparable to o1
In January 2025, Moonshot AI released Kimi k1.5, a multimodal large model trained with reinforcement learning (RL) without Monte Carlo tree search, value functions, or process reward models. Through long-context scaling and improved policy optimization methods, it achieved 77.5 on AIME, 96.2 on MATH 500, 94th percentile on Codeforces, and 74.9 on MathVista, matching OpenAI o1. It also proposed the long2short method, using long chain-of-thought to improve short chain-of-thought models, achieving 60.8 on AIME, 94.6 on MATH 500, and 47.3 on LiveCodeBench, significantly surpassing GPT-4o and Claude Sonnet 3.5 (up to 550% improvement).
Kimi k1.5 demonstrates that reinforcement learning can serve as a new axis for scaling large models, breaking the limitation of relying solely on pretraining data. Its core contribution lies in simplifying the RL framework by removing complex components (e.g., MCTS, value functions) while achieving SOTA through long context and policy optimization. The long2short method further provides a practical path for transferring knowledge from long chain-of-thought models to short models, directly impacting inference efficiency and cost optimization.
Kimi k1.5 adopts an RL training framework based on a streaming diffusion Transformer. Key innovations include: 1) Long-context scaling, supporting longer reasoning chains to enhance complex reasoning; 2) Improved policy optimization methods, such as PPO variants, stabilizing training and improving sample efficiency. The model achieves or surpasses o1 on multiple math, programming, and multimodal benchmarks without search or process rewards. The long2short method compresses the capabilities of long chain-of-thought models into short models via knowledge distillation, balancing inference speed and accuracy. Limitations: RL training is sensitive to reward design, and long-context reasoning incurs high computational costs.
The release of Kimi k1.5 marks that Chinese AI companies have reached international top-tier levels in multimodal reasoning, directly challenging OpenAI o1. Its open-source strategy (code and weights publicly available) will accelerate industry applications, especially in scenarios requiring deep reasoning such as mathematics, programming, and scientific computing. The long2short method provides a cost-effective solution for model deployment, potentially driving more enterprises to adopt RL training rather than solely scaling pretraining.
It is recommended that AI infrastructure teams evaluate Kimi k1.5's RL framework for internal model training, especially for math and programming tasks. Investment opportunities include Moonshot AI and its ecosystem partners. On the engineering side, the long2short method can be adopted to optimize existing reasoning models, reducing latency and cost.
Areas to watch: 1) Scalability of RL training, including automation of reward design and hardware efficiency for long-context reasoning; 2) Generalization of the long2short method to other domains (e.g., code generation, scientific reasoning); 3) Model safety and alignment, as RL training may introduce unexpected behaviors.