Beyond Converging Representations: A Philosophical Response on the Interpretation Risks of Scientific Foundation Models: Quantitative Benchmark and Generality Boundaries of Representation Alignment in Scientific Foundation Models
In December 2025, a study analyzed the internal representations of nearly 60 scientific models (covering string, graph, 3D atomic, and protein modalities) and found they are highly aligned on chemical systems. High-performing models exhibit consistent representations within the training distribution, while weak models diverge; however, on out-of-distribution structures, almost all models collapse to low-information representations. The study proposes representation alignment as a quantitative benchmark for the generality of scientific models, which can be used for model selection and distillation.
This study systematically demonstrates for the first time that scientific models of different modalities and architectures (e.g., molecular, material, and protein prediction models) learn highly consistent representations of physical reality within the training distribution, but all fail during out-of-distribution generalization. This finding provides a quantitative metric—representation alignment—for the generality of scientific foundation models, and reveals that current models are still limited by training data and inductive biases, and have not yet encoded truly universal physical laws. The research offers new tools for model selection, distillation, and cross-modal transfer, marking a milestone in the AI for Science field.
The study analyzed nearly 60 scientific models, including strings (e.g., SMILES), graph neural networks, 3D atomic potential functions, and protein models, comparing their internal representations using representation alignment metrics (e.g., CCA, Procrustes). Key findings: Within the training distribution, high-performing models (e.g., machine learning interatomic potentials) converge in representation as performance improves, while weak models diverge to local suboptima; but on out-of-distribution data (e.g., novel molecular structures), almost all model representations collapse to low-information states, indicating a lack of true generality. The study also identifies two distinct mechanisms: high alignment within the training distribution and low-information collapse out of distribution. This method can serve as a quantitative benchmark for evaluating the generalization ability of scientific foundation models and guide model distillation (selecting models with high representation alignment for transfer).
This research has direct guiding significance for the AI for Science industry: representation alignment can serve as a criterion for scientific model selection, helping enterprises and research institutions filter models with stronger generalization ability. In fields such as drug discovery and materials design, the risk of model failure in out-of-distribution scenarios needs to be incorporated into evaluation. Additionally, the distillation method proposed in the study can reduce computational costs, promoting the deployment of scientific models in industry.
It is recommended that AI pharmaceutical and materials science companies adopt representation alignment as a supplementary metric for model evaluation, prioritizing models that maintain high alignment out of distribution. Meanwhile, model distillation tools can be developed based on this research to compress knowledge from multiple specialized models into a single efficient model, reducing inference costs. For investment institutions, focus on technology transfer companies in this direction.
Attention should be paid to whether representation alignment can predict model performance on real scientific tasks (e.g., catalyst design, protein folding). Future research should explore how to improve out-of-distribution representation quality through multimodal training or physical constraints. Furthermore, the automation and tooling of this benchmark will accelerate its application in industry. In terms of safety, the unreliability of models in out-of-distribution scenarios may pose risks of erroneous scientific conclusions.