Into the ORBIT for Time Series: Training Regimes for Foundation Models
A paper titled 'Into the ORBIT for Time Series: Training Regimes for Foundation Models' was published on arXiv on 2026-08-13. It introduces ORBIT (Omni-Range Bootstrap Incremental Training), a training paradigm for time series foundation models that combines Bootstrap Multi-Level Sampling and Omni-Range Incremental Training. The paper also presents Falcon-2.0, a univariate encoder-only Transformer with missingness-aware triple-channel patch tokenization and parallel patch prediction, and Rank-Guided Cross-Depth Alignment, a training objective using late-layer representations as stop-gradient teachers for shallow layers. Evaluations on GIFT-Eval and fev-bench are mentioned.
The paper addresses the under-explored area of training regimes for time series foundation models, focusing on controlling pre-training distributions with respect to domain imbalance, context requirements, prediction horizons, and missingness. ORBIT makes this distribution explicit and controllable. Falcon-2.0 is trained under ORBIT. The paper reports strong zero-shot performance on GIFT-Eval and fev-bench.
ORBIT introduces a controllable pre-training distribution via Bootstrap Multi-Level Sampling and Omni-Range Incremental Training. Falcon-2.0 uses missingness-aware triple-channel patch tokenization and parallel patch prediction. Rank-Guided Cross-Depth Alignment uses late-layer representations as stop-gradient teachers for shallow layers without additional inference cost.
The work suggests a shift from purely architectural innovation to training regime design for time series foundation models, potentially improving zero-shot performance across heterogeneous time series data.
Improved training regimes for time series foundation models could lead to more robust and generalizable models for forecasting and anomaly detection across industries such as finance, energy, and healthcare.
Observable next signals include follow-up papers applying ORBIT to other architectures, adoption of Falcon-2.0 in time series benchmarks, and further evaluation on additional datasets beyond GIFT-Eval and fev-bench.