InternLM released a Qwen3-based model that grades mathematical proofs
InternLM (Shanghai AI Laboratory) released AdvancedMathBench-AutoVerifier on Hugging Face, a Qwen3 5MoE-based model that evaluates natural-language mathematical proofs, explains errors, and identifies the earliest incorrect step. Weights are 40 safetensors shards (~68 GiB), and the tokenizer requires sentencepiece and trust_remote_code=True.
China context
- Original name
- 上海人工智能实验室
- Outside China
- Open weights · huggingface.co
- Claims
- Company-reported; not yet independently evaluated
- For builders
- Developers outside China can download the model weights from Hugging Face and integrate the proof verifier into their own pipelines, but must handle the tokenizer's trust_remote_code requirement and the model's learned-grader limitations.
- For investors
- The release signals continued open-weight contributions from Shanghai AI Laboratory in specialized AI evaluation tools, but the model's niche focus and lack of independent validation may limit immediate commercial impact.
InternLM (Shanghai AI Laboratory) released AdvancedMathBench-AutoVerifier on Hugging Face. The model evaluates natural-language mathematical proofs, explains errors, and identifies the earliest incorrect step. It serves as the automatic grader for AdvancedMathBench's ProverBench. The model is based on Qwen3 5MoEForConditionalGeneration, with weights in 40 safetensors shards (~68 GiB). The tokenizer is bundled InternS1Tokenizer and requires sentencepiece and trust_remote_code=True after reviewing the tokenizer code. Input uses proof_verifier.md with a problem, optional reference solution, and candidate proof split into zero-indexed steps. Output includes an assessment, identified errors, and the first error index (-1 means no error). ProverBench checks each proof 8 times and accepts only when all 8 judgments report -1. AutoVerifier is a learned grader, not a formal proof checker, and can make errors.
The model is a Qwen3 5MoEForConditionalGeneration with 40 safetensors shards (~68 GiB). It uses a bundled InternS1Tokenizer requiring sentencepiece and trust_remote_code=True. Input format is proof_verifier.md with problem, optional reference solution, and candidate proof split into zero-indexed steps. Output includes assessment, identified errors, and first error index (-1 means no error). ProverBench checks each proof 8 times and accepts only when all 8 judgments report -1. AutoVerifier is a learned grader, not a formal proof checker, and can make errors.
Researchers and developers working on mathematical reasoning benchmarks can now use a released model to automatically grade natural-language proofs, reducing manual evaluation effort for proof-generation systems. The model's learned-grader nature means its judgments are not formally verified, so users must validate its error identification for their own use cases.
The release provides a tool for automatic grading of mathematical proofs, which could reduce evaluation costs for research groups working on theorem proving and mathematical reasoning. However, the model's learned nature and potential for errors mean it may require human oversight in high-stakes applications.
Observable next signals include whether the model is adopted in other proof benchmarks, whether the ProverBench dataset is released, and whether the model's error identification is independently evaluated.