TabPFN · Jul 28, 2026
Empirical Evaluation of Out-Of-Distribution Performance of Tabular Foundation Models
A study evaluated nine tabular foundation models (TFMs) on three real-world datasets with distribution shifts. All models showed systematic degradation under shift, with gaps from 0.003 to 0.060. The relationship between in-distribution and OOD performance for classical models extends to TFMs.
What happened
An empirical evaluation assessed the out-of-distribution (OOD) performance of nine tabular foundation models (TFMs) including TabPFNv2, TabPFNv2.5, TabPFNv2.6, TabPFNv3, TabICL, TabICLv2, Mitra, LimiX, and TabFM. Using three datasets from the TableShift study (HELOC, Voting, Childhood Lead) covering label, socioeconomic, and geographic shifts, the study found that all TFMs degrade systematically under distribution shift, with shift gaps ranging from 0.003 to 0.060. The documented relationship between in-distribution and OOD performance for classical tabular models also holds for TFMs.
Technical significance
The study reveals that current TFMs lack robustness to distribution shifts, with performance degradation consistent across diverse pre-training strategies and architectures. This suggests that existing pre-training methods do not inherently capture invariant representations for tabular data under shift.
Industry impact
For real-world deployment of TFMs in sectors like finance or healthcare, where distribution shifts are common, the observed OOD degradation implies that relying solely on in-distribution benchmarks may overestimate model reliability. Practitioners should incorporate shift-aware evaluation and consider hybrid approaches with robust classical models.
What to watch
Future research may focus on developing TFMs with built-in robustness to distribution shifts, possibly through domain-invariant pre-training objectives or shift-aware fine-tuning. Monitoring the release of new TFM versions with claimed OOD improvements will be a key signal.
Decision value
The findings highlight a risk for enterprises adopting TFMs for critical tabular tasks, as performance can degrade under realistic shifts. This creates an opportunity for tools and services that benchmark and improve TFM robustness, or for hybrid solutions that combine TFMs with traditional robust models.