Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM
A research paper on arXiv (2609.04098v1) reports that Minima, an NVFP4 W4A4 quantization applied to all 496 linear layers of a hybrid 27B LLM including Gated DeltaNet (GDN) layers, matches BF16 within seed noise across perplexity at 4K/32K, MMLU-Pro, GSM8K, AIME'25, GPQA-Diamond, LiveCodeBench, and RULER retrieval to 64K. The 5-task average difference is -0.52. Minima is the smallest recipe at 17.5 GiB and fastest prefill (+14-19%). The 32K perplexity gap shrinks with position. The paper includes a four-part mechanism study.
Hybrid LLMs pair softmax attention with linear-attention layers such as Gated DeltaNet (GDN), whose recurrent state summarizes context in fixed size. Early community 4-bit quantizations of Qwen3.8-27B (48 GDN layers, 16 attention layers) left the GDN block in 8- or 16-bit precision, especially its decay and write-strength gates, on the intuition that errors in a recurrence accumulate over long contexts. The paper tests that intuition by building Minima: NVFP4 W4A4 on all 496 linear layers, GDN included. Across perplexity at 4K/32K, MMLU-Pro, GSM8K, AIME'25, GPQA-Diamond, LiveCodeBench, and RULER retrieval to 64K, Minima matches BF16 within seed noise (5-task average -0.52) while being the smallest (17.5 GiB) and fastest-prefill (+14-19%) recipe compared, and its 32K perplexity gap shrinks with position. A four-part mechanism study explains why: (i) NVFP4's 16-element block scaling localizes the residual stream's extreme outliers, equalizing activation error across layer roles; (ii) the supposedly fragile gate projections are the least sensitive -- softplus/exponential and sigmoid parameterizations compress ~11% GEMM error to ~2% output error; (iii) the delta-rule recurrence holds...
NVFP4's 16-element block scaling localizes extreme outliers in the residual stream, equalizing activation error across layer roles. Gate projections using softplus/exponential and sigmoid parameterizations compress ~11% GEMM error to ~2% output error, making them less sensitive than expected. The delta-rule recurrence's error behavior is not fully detailed in the provided evidence, but the 32K perplexity gap shrinking with position suggests error accumulation is mitigated.
The result challenges the common practice of keeping recurrent layers at higher precision in hybrid LLMs, potentially enabling more aggressive quantization for deployment. This could reduce memory footprint and increase inference speed for models with linear attention, making them more viable for edge or cost-sensitive applications.
Minima's 17.5 GiB size and +14-19% faster prefill could lower serving costs and hardware requirements for hybrid LLMs, enabling deployment on smaller GPUs or edge devices. This may expand the market for efficient long-context models.
Observable next signals include further studies on the delta-rule recurrence error behavior, adoption of NVFP4 W4A4 in production hybrid LLMs, and benchmarks on longer contexts beyond 64K. Replication by other research groups and integration into quantization toolkits would validate the findings.