LACE: Layer-Wise Compression for Dynamic Frame Rate Codecs
LACE (Layer-Adaptive Codec Encoding) is a dynamic frame rate neural audio codec that applies an independent compression step at each quantization layer, enabling layer-specific segmentation boundaries. It introduces union alignment and boundary anchor mechanisms for downstream text-to-speech (TTS). Experiments on LibriTTS show LACE offers a better rate-quality tradeoff than prior dynamic frame rate methods on reconstruction and improves TTS inference.
Neural audio codecs are key for speech language modeling but high frame rates increase computational costs. Dynamic frame rate codecs reduce effective frame rate by merging frames, but prior methods either use single-codebook codecs or a single compression step before multi-layer quantization, forcing all layers to share segmentation boundaries despite different rates of change. LACE applies independent compression per quantization layer, with union alignment and boundary anchors for TTS. On LibriTTS, LACE achieves better rate-quality tradeoff and improved TTS inference.
LACE decouples compression across quantization layers, allowing each residual embedding layer to have its own segmentation boundaries. Union alignment and boundary anchor mechanisms reconcile layer-specific durations for TTS, preserving compression benefits while maintaining consistency. This suggests a shift toward layer-adaptive temporal modeling in neural codecs.
Improved rate-quality tradeoff in audio codecs can reduce sequence lengths for speech language models, lowering inference costs and latency. This may accelerate adoption of neural audio codecs in real-time TTS and voice AI products, particularly where computational efficiency is critical.
LACE's efficiency gains could reduce serving costs for speech AI systems, enabling more scalable voice interfaces and lower-latency TTS. It may create competitive advantage for companies deploying large-scale speech models.
Next signals include open-source release of LACE code or checkpoints, integration into speech language model pipelines, and comparative benchmarks on other datasets beyond LibriTTS. Adoption by TTS frameworks or commercial voice platforms would indicate practical impact.