Unifying ICL, SFT, KL-Regularized RL Through a Bayesian Lens
A paper published on arXiv on 2026-09-04 proposes a Bayesian framework that unifies in-context learning, supervised fine-tuning, and KL-regularized reinforcement learning as forward-KL projections onto a posterior.
The paper develops a Bayesian perspective that places supervised fine-tuning, few-shot in-context learning, KL-regularized RLHF/RLVR, on-policy distillation, and test-time reasoning on the same footing. It uses a two-step template: construct a Gibbs posterior using a prior and utility signal, then approximate it via forward-KL projection either in-weights or in-context.
The core technical contribution is formalizing few-shot ICL and SFT as amortized and in-weights projections onto the Bayes posterior predictive, and showing that KL-regularized RLHF/RLVR, reward-weighted SFT, reward-weighted ICL, and advantage-weighted SFT are all instances of forward-KL projection onto posterior distributions.
This unification may help explain recent empirical results such as the mixed impact of few-shot prompting on RL-tuned reasoning models, and could guide more principled selection of training paradigms.
A unified theoretical framework could reduce trial-and-error in choosing between SFT, ICL, and RL methods, potentially lowering development costs and improving model alignment efficiency.
Observable next signals include follow-up empirical studies testing the Bayesian framework's predictions on RL-tuned reasoning models, and potential adoption of the framework in training pipelines.