Can LLMs Design Video Coding Tools? A Case Study on Planar Mode
A paper on arXiv (cs.AI) explores whether large language models can design video coding tools, using a case study on Planar mode. Experiments in a generation-and-evaluation loop show an LLM-generated mode outperforms the conventional Planar mode in Fraunhofer Versatile Video Encoder (VVenC) faster preset, achieving 0.18% bitrate savings with 0.4% complexity overhead. The study also extends to Enhanced Compression Model (ECM) with directional Planar modes.
The paper investigates the use of large language models to design video coding tools, focusing on Planar mode. Through iterative generation and evaluation, the LLM-generated predictor achieved 0.18% bitrate savings with 0.4% complexity overhead compared to the default Planar mode in VVenC. The research also explores integration strategies in ECM, including replacing directional Planar modes or adding the LLM-generated predictor as an additional mode.
The generation-and-evaluation loop enables LLMs to iteratively refine video coding tools based on encoder performance feedback. The LLM-generated Planar predictor achieves bitrate savings with minimal complexity overhead, suggesting LLMs can optimize algorithmic components in video codecs. Extension to ECM with directional Planar modes indicates potential for broader application in advanced codecs.
This research signals a potential shift toward AI-assisted design of video coding tools, which could accelerate codec development and optimization. If LLM-generated tools prove robust, they may reduce manual engineering effort and lead to more efficient video compression standards, impacting streaming, storage, and communication industries.
Improved video compression efficiency can reduce bandwidth and storage costs for streaming services, video platforms, and communication providers. LLM-assisted codec design could lower R&D costs and time-to-market for new compression technologies, creating competitive advantages for companies investing in AI-driven codec development.
Next observable signals include publication of detailed results for ECM integration strategies, replication by other codec research groups, and potential adoption of LLM-generated tools in future video coding standards or open-source encoders. Further research may explore LLM design for other coding tools beyond Planar mode.