Can Large Language Models Recover Semantic Optimization Opportunities That Compilers Miss?
A study introduces SeGaBench, a benchmark of 120 C/C++ cases (100 synthetic, 20 source-backed) where compilers miss optimizations due to absent semantics. Five LLMs were evaluated; the best model produced correct artifacts in 94.8% of responses, achieved at least 1.05x speedup in 83.3%, and succeeded on 93.3% of cases, though often only partially closing the oracle performance gap.
Researchers created SeGaBench to test whether LLMs can recover hidden semantic information that compilers fail to exploit, enabling missed optimizations. The benchmark includes cases with low-level assumptions, data-structure invariants, and high-level semantic lifting. The top-performing LLM generated correct, validated artifacts in most cases, demonstrating that LLMs can serve as speculative semantic proposers to complement traditional compiler analysis, provided their outputs are validated.
LLMs can infer missing semantics from heterogeneous C/C++ context and propose contract-preserving code transformations that compilers overlook. The approach relies on a validation pipeline with correctness and semantic checks, and a reproducible performance protocol. The gap between correct artifacts and full oracle performance suggests that LLM proposals often capture only a subset of the possible optimization.
This work points to a new role for LLMs in compiler toolchains as semantic assistants, potentially improving performance of legacy or complex codebases without manual annotation. Adoption would require robust validation frameworks to ensure safety and correctness, but could reduce the need for expert-driven compiler hints or domain-specific languages.
If integrated into development workflows, LLM-driven semantic optimization could yield performance improvements in critical software with minimal developer effort, reducing compute costs and energy consumption. This could be particularly valuable for high-performance computing, embedded systems, and cloud services where even small speedups translate to significant savings.
Next signals include integration of LLM-based semantic proposers into production compilers, development of more comprehensive benchmarks, and research into closing the remaining performance gap. Further work may explore fine-tuning LLMs on compiler-specific semantics or combining them with formal methods for stronger guarantees.