SSTQ: Privacy-Preserving Vector Quantization via Subsampled Stochastic TurboQuant
Researchers propose Subsampled Stochastic TurboQuant (SSTQ), a framework for local differential privacy in distributed optimization. SSTQ combines overcomplete equal-norm tight frames, coordinate subsampling, and privacy-aware one-dimensional quantization. Two variants are introduced: Flat Randomized Response and Metric-Aware Laplace. The method uses ⌈log₂ N⌉ + b bits per client, with N = Θ(d). A surrogate privacy-aware codebook objective reduces codebook-dependent MSE scaling from O(4^b) to O(2^b). Empirical evaluation on CIFAR-10 and Fashion-MNIST federated learning tasks shows favorable utility and communication efficiency.
SSTQ is a new vector quantization method for federated learning that provides local differential privacy with low communication cost. It uses overcomplete frames and subsampling to achieve optimal mean squared error scaling while using only ⌈log₂ N⌉ + b bits per client. The framework includes two variants and a novel codebook objective that improves privacy-utility trade-offs. Experiments on CIFAR-10 and Fashion-MNIST demonstrate its effectiveness against baselines.
SSTQ addresses the dimension-dependent variance of prior geometric quantization methods by combining overcomplete equal-norm tight frames with coordinate subsampling. The surrogate privacy-aware codebook objective is a key innovation, halving the exponent in the codebook-dependent MSE scaling from 4^b to 2^b, which significantly improves utility for higher bit-widths. The Metric-Aware Laplace variant is designed for higher codebook bit-width regimes, suggesting a nuanced approach to privacy-utility trade-offs.
This research targets the growing need for privacy-preserving federated learning in industries like healthcare and finance, where data cannot be centralized. By reducing communication bits while maintaining differential privacy, SSTQ could lower bandwidth costs and enable on-device learning on edge devices. The two variants offer flexibility for different privacy budgets and accuracy requirements, potentially accelerating adoption of federated learning in sensitive applications.
SSTQ could reduce the communication overhead of federated learning by up to 50% compared to existing quantization methods while providing formal differential privacy guarantees. This makes it attractive for enterprises deploying federated learning on mobile devices or across bandwidth-constrained environments. The improved codebook objective may also lower the computational cost of quantization, further reducing total cost of ownership.
Next signals to watch include: (1) open-source implementation release, (2) follow-up work extending SSTQ to other distributed learning paradigms like split learning, (3) real-world deployment in privacy-sensitive sectors, and (4) theoretical analysis of the privacy-utility-communication trade-off under different threat models. The method's performance on larger models and datasets will be critical for practical adoption.