Efficient Test-Time Adaptation through Human-AI Interaction
A research paper proposes test-time adaptation through human-agent interaction (TAHI), which integrates cross-session interaction data into agent context and weights, and uses an evolving rubric module to crystallize user criteria. The method was tested with 30 individuals across writing and visual creation domains on 600 tasks, improving solo task success by 4.5-20.9% within tens of tasks.
The paper introduces TAHI, a method for adapting AI agents to individual users by leveraging iterative human-agent interaction data. It uses an evolving rubric module to capture user-specific evaluation criteria. Experiments with 30 users on 600 tasks in writing and visual creation show significant improvements in task success rates.
TAHI integrates interaction signals into both agent context and weights, and employs a dynamic rubric module that evolves with user feedback. The approach demonstrates that test-time adaptation can effectively personalize agent behavior with limited interaction data, achieving notable gains in task success.
This research highlights the potential for personalized AI agents that adapt to individual user preferences and expertise, which could enhance user satisfaction and adoption in creative and professional tools.
Personalized AI agents could increase user retention and productivity in applications like writing assistants and design tools, offering a competitive edge for companies that implement such adaptation techniques.
Future work may explore scaling TAHI to larger user bases, extending to other domains, and refining the rubric module for more complex evaluation criteria. Observing adoption in commercial AI products could signal practical viability.