Baichuan 2: Open Large-scale Language Models
Baichuan Intelligence released the Baichuan 2 series of large language models, including 7B and 13B parameter sizes, trained from scratch on 2.6 trillion tokens. They match or surpass open-source models of the same size on public benchmarks such as MMLU, CMMLU, GSM8K, and HumanEval, and perform excellently in vertical domains like healthcare and law. All pre-trained model checkpoints will be open-sourced.
Baichuan 2 is the second-generation open-source large language model from Baichuan Intelligence. Through multilingual pre-training on 2.6 trillion tokens, it achieves leading performance in both general and vertical domains. Its open-source strategy lowers the barrier for enterprise deployment, especially suitable for Chinese scenarios and industry customization.
Baichuan 2 adopts the standard Transformer architecture, with training data including Chinese, English, and multilingual corpora totaling 2.6 trillion tokens. The model performs well on benchmarks such as MMLU (English knowledge), CMMLU (Chinese knowledge), GSM8K (math reasoning), and HumanEval (code generation), comparable to or better than open-source models like Llama 2. In healthcare and legal domains, fine-tuning with domain data can further improve performance. Model weights are fully open-sourced, supporting reproduction and secondary development by the research community.
Baichuan 2's open-source strategy will accelerate the application of large models in the Chinese internet and vertical industries, especially benefiting fields requiring specialized knowledge such as healthcare and law. Its competition with models of similar size may promote diversification of the open-source ecosystem and reduce enterprises' reliance on closed-source models.
Enterprises can deploy Baichuan 2 privately based on its open-source weights for scenarios such as intelligent customer service, document analysis, and code assistance. It is recommended to trial it in Chinese-dominated businesses first and evaluate the cost and benefit of fine-tuning in specific vertical domains.
Future attention should be paid to the model's performance on more Chinese benchmarks (e.g., CLUE) and the effectiveness of domain-specific fine-tuned models developed by the community based on open-source weights. Additionally, model safety and bias control are important directions for verification.