QuoteBench measures LLM coding agents' Bash command execution across generation and execution transport boundaries using 56 one-shot tasks from 14 incident-derived families. It introduces one deliberately unescaped added parser. Replaying the same reply through the added parser lowers success by 55.4 to 73.2 percentage points across eight same-window configurations. Disclosure recovers 30.4 to 60.7 points for six configurations, and zero or slightly negative for the other two. GPT-5.6-sol's matched gap of -3.6 points hides -64.3 points of damage and +60.7 points of compensation. The deployment configuration reorders models, with one reversal among 26 comparable pairs.
LLM coding agents issue Bash commands through interfaces that may serialize, wrap, and reparse model output. Matched execution scores alone cannot distinguish command-generation errors from failures introduced after generation. QuoteBench measures this boundary with exact final-state validation on 56 one-shot tasks from 14 incident-derived families, crossing the generation contract with the execution transport around one deliberately unescaped added parser. Escaping at the interpolation point reproduces each replayed reply's raw-path outcome, so any recovery under a disclosed boundary must come from the model changing its generation. Across eight same-window configurations, replaying the same reply through the added parser lowers success by 55.4 to 73.2 percentage points; disclosure recovers 30.4 to 60.7 points for six configurations, and zero or slightly negative for the other two. Raw generation is nearly saturated at the frontier; boundary adaptation is what still separates models. GPT-5.6-sol's matched gap of -3.6 points hides -64.3 points of damage and +60.7 points of compensation. The deployment configuration reorders models: one reversal among 26 comparable pairs is unambig…
The benchmark isolates the execution transport boundary by replaying identical model replies through an added unescaped parser, showing that success drops of 55.4–73.2 percentage points are attributable to post-generation serialization and reparsing rather than model generation. Disclosure of the boundary recovers 30.4–60.7 points for six of eight configurations, indicating that models can adapt their command generation when the transport contract is known. Raw generation is nearly saturated at the frontier, so remaining model differences are driven by boundary adaptation rather than core command synthesis.
Current matched execution scores systematically overstate real-world reliability of LLM coding agents because they conflate generation quality with transport robustness. Deployment configuration can reorder model rankings, as evidenced by one reversal among 26 comparable pairs, meaning that benchmark leaderboards may not reflect production performance. This creates a risk for enterprises adopting coding agents based on standard benchmarks without testing their specific command execution stack.
For enterprises deploying LLM coding agents, QuoteBench provides a method to quantify hidden failure modes in command execution that standard benchmarks miss. The observed 55.4–73.2 point success drop under an unescaped parser implies significant operational risk and potential remediation cost. Disclosing the execution boundary can recover 30.4–60.7 points, suggesting that configuration transparency and boundary-aware prompting can materially improve agent reliability without model changes.
Expect increased focus on boundary-aware evaluation and disclosure of execution transport contracts in agent benchmarks. Model developers may begin reporting both matched and boundary-adjusted scores. Tooling that validates command serialization and escaping at the interpolation point could become a standard part of agent deployment pipelines. If raw generation remains saturated, competitive differentiation will shift to robust boundary adaptation and integration with diverse execution environments.