Design Docs Are All You Need: An AI-native Machine-Learning Performance Tool
SMART is a symbolic performance-modeling library for ML systems whose main branch contains almost no code; the repository is a DAG of self-contained natural-language design docs, coding sub-agents regenerate the implementation from only the docs on new version updates, and every human change is a natural-language edit to a doc. Two ingredients make regeneration reliable: a design-doc style built around step-by-step worked examples that act as in-context demonstrations for the generating agents, and a minimal, recursively defined operator IR with symbolic (SymPy) cost expressions, a fast analytical roll-up mode for large sweeps, and a slow modulo-scheduling mode for fine-grained schedule studies.
Machine-learning performance modeling is a uniquely hostile terrain for long-lived software: the assumptions baked into today's abstractions are invalidated by tomorrow's models and systems, forcing perpetual refactoring of performance-modeling frameworks. Meanwhile, AI coding agents have become fast and capable enough that regenerating an entire library is cheaper than paying down the tech debt of incrementally patching it. SMART is a rigorous symbolic performance-modeling library for ML systems whose main branch contains almost no code: the repository is a DAG of self-contained natural-language design docs, coding sub-agents regenerate the implementation from only the docs on new version updates, and every human change is a natural-language edit to a doc--self-documenting by construction.
The approach relies on two key technical mechanisms: (1) a design-doc style built around step-by-step worked examples that serve as in-context demonstrations for the generating agents, and (2) a minimal, recursively defined operator IR with symbolic (SymPy) cost expressions, a fast analytical roll-up mode for large sweeps, and a slow modulo-scheduling mode for fine-grained schedule studies. This enables regeneration of the entire library from natural-language docs on version updates.
The work signals a shift toward AI-native software development where code is treated as a regenerable artifact derived from design documentation, potentially reducing long-term maintenance costs in rapidly evolving domains like ML performance modeling. It suggests that AI coding agents are now capable enough to make regeneration cheaper than incremental patching.
For organizations building ML systems, SMART could reduce engineering effort spent on maintaining performance-modeling tools, enabling faster adaptation to new models and hardware. The doc-as-source-of-truth approach may also improve knowledge retention and onboarding, though the business value is not yet quantified in the evidence.
Observable next signals include adoption of similar doc-driven regeneration approaches in other performance-sensitive or fast-moving software domains, further improvements in coding agent reliability for complex symbolic libraries, and potential open-source releases or benchmarks comparing SMART against traditional performance-modeling frameworks.