Interaction Scaling: Grounding the Third Axis of Test-Time Compute
The paper "Interaction Scaling: Grounding the Third Axis of Test-Time Compute" proposes interaction as the third axis of test-time compute, where external instruments observe model-generated artifacts and provide feedback, enabling the model to revise based on real observations. Experiments show that on hard-coded tasks with a fixed token budget, both reasoning and best-of-N sampling methods plateau, while interaction strategies continue to improve, with the proposer-reviewer method achieving 100% pass rate with zero variance, consistent across three model families.
The paper proposes interaction as the third axis of test-time compute, using external instruments to provide real observation feedback, breaking through the performance ceiling of reasoning and sampling methods. On hard-coded tasks, interaction strategies continuously improve, with the proposer-reviewer method achieving 100% pass rate.
Interaction scaling introduces external instruments to provide real observations, allowing the model to revise based on actual behavior, thereby breaking the information bottleneck of internal reasoning and sampling. The proposer-reviewer method achieves perfect pass rate under a fixed token budget, indicating significant advantages of interaction strategies on coding tasks.
This research may drive a shift in AI systems from purely internal reasoning to interactive reasoning, particularly in domains requiring external validation (e.g., code generation, vision tasks).
Interaction scaling methods may improve the reliability of AI systems in critical tasks (e.g., code generation), reduce manual review costs, and have commercial application value.
Future work could explore the application of interaction scaling methods to broader tasks (e.g., vision, robotics) and the impact of external instrument design on performance.