Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs
A paper titled 'Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs' was published on arXiv on 2026-09-09. It proposes FOM-UL, a layer-level unlearning framework that selects transformer layers using a forget-to-retain significance score to improve forgetting-utility trade-off and quantization resilience.
The paper introduces FOM-UL, a layer-selective unlearning method for large language models. It targets layers with high influence on forget data and low sensitivity to retain data, reducing residual memorization while preserving utility and improving robustness to post-training quantization.
FOM-UL uses a forget-to-retain significance score to select layers for unlearning, concentrating updates where they are most effective. This reduces the chance that small, diffuse updates are erased by low-bit rounding, providing an empirical path toward quantization-resilient unlearning.
The approach addresses practical deployment concerns for LLM providers facing privacy and copyright removal requests, as it aims to maintain model utility while making unlearning more robust to common post-training optimizations like quantization.
Layer-selective unlearning could lower the cost and risk of removing sensitive data from deployed LLMs, supporting regulatory compliance and reducing the need for expensive full retraining.
Observable next signals include follow-up research validating FOM-UL on larger models or different quantization schemes, and potential adoption by model vendors seeking compliant unlearning without full retraining.