Cross-Layer Interaction under Weight-Space Ablation: A Closed-Form Attention Jacobian Bound and a Test on a Real Pretrained Model
A research paper extends prior work on weight-space ablation and activation patching agreement. It derives an exact decomposition of cross-layer interaction into same-block terms and a cross-layer remainder for two layers, expressed as a double integral of a mixed second derivative. A closed-form Jacobian bound for the attention sub-block is derived and verified without violation against Qwen2.5-1.5B-Instruct's real weights.
This paper generalizes a previous idealized model of conditional computation in residual streams to multi-layer settings. The interaction from ablating carriers across layers is decomposed into per-layer same-block terms and a cross-layer remainder. For two layers, the remainder is isolated as a double integral requiring an attention Jacobian bound. The authors derive this bound in closed form and empirically validate it on the pretrained Qwen2.5-1.5B-Instruct model, finding no violations.
The key technical contribution is a closed-form bound on the Jacobian of the attention sub-block, enabling rigorous analysis of cross-layer interactions under weight-space ablation. The decomposition shows that cross-layer effects can be expressed as a double integral of a mixed second derivative, with the bound ensuring smallness under certain conditions.
This work advances mechanistic interpretability by providing mathematical tools to analyze how ablations in one layer affect others in real pretrained models. It strengthens the theoretical foundation for techniques like activation patching, which are used to understand model behavior and could inform safer and more reliable AI systems.
Improved interpretability methods can reduce risks in deploying large language models by making their internal computations more transparent. This may lower barriers for enterprise adoption in regulated industries where explainability is required.
Next signals include application of the Jacobian bound to larger models, extension to more than two layers, and integration into interpretability tooling. The closed-form nature may enable automated verification of ablation consistency in production models.