Spreadsheet-RL: Table Agents Shift from Prompt Engineering to Reinforcement Learning in Real Excel Environments
Submitted on May 21, 2026, Spreadsheet-RL builds a real Microsoft Excel multi-turn reinforcement learning environment with financial and supply chain task sets; the paper reports that Qwen3-4B-Thinking-2507 improves Pass@1 from 12.0% to 23.4% on SpreadsheetBench and from 8.4% to 17.2% on Domain-Spreadsheet.
Spreadsheets are one of the most common data work interfaces in enterprises, but general models with prompting remain unstable in multi-step operations. This study connects real files, tool routing, task outcomes, and reinforcement learning, signaling that office agents are moving toward specialized training.
The framework includes a data pipeline that constructs starting and target tables from public forums, a Python sandbox covering Excel functions, refined tool routing rules, and a multi-turn Spreadsheet Gym, using executable results to perform reinforcement learning on policies.
Competition in office agents will shift from demo-style interface operations to task environments, verifiable rewards, and vertical training data; financial and supply chain spreadsheets may become the earliest enterprise entry points for specialized agents.
Enterprises should first select spreadsheet processes that are rollback-capable and result-verifiable for training and evaluating agents, and incorporate manual review, failure recovery, and cost per correct task into ROI.
Although relative improvements are significant, absolute pass rates remain below production requirements; reliability with complex formulas, external data, permission controls, error recovery, and cross-version Excel needs verification.