Does Your Agent's Memory Survive a Model Upgrade? A Controlled Study of Memory Portability
A controlled study compares memory portability across model upgrades using 48 synthetic histories, exact scoring, and two open-weight models under 10B parameters. Fixed-schema knowledge graph (KG-fixed) accuracy changes by only +0.0004 ± 0.0020 after a writer swap, while compressed natural-language notes (NOTES) shift asymmetrically by +9.91 or -13.28 percentage points depending on migration direction. Partial embedding migrations with a 50/50 mixed index capture only a 4.96-point accuracy improvement versus an 11.90-point gain from full migration.
Model upgrades are routine but memory migrations are not. An agent can keep the same memory store and still forget: a new model may interpret old notes differently, mixed embedding versions may break retrieval, and repair may fail without original evidence. The study compares memory as verbatim long-context reading (LC-RAW), chunked retrieval-augmented generation (RAG), compressed natural-language notes (NOTES), or normalized fixed-schema knowledge graph (KG-fixed). Fixed-schema structures transfer reliably, while compressed notes exhibit high model coupling. In RAG systems, partial embedding migrations capture only a fraction of the accuracy gain achieved through full migration.
Fixed-schema knowledge graphs provide stable memory representation across model upgrades, with near-zero accuracy drift. Compressed natural-language notes are highly sensitive to the writer model, causing asymmetric accuracy shifts. Mixed embedding indexes degrade retrieval performance, suggesting that full embedding migration is necessary to realize most accuracy gains.
Memory portability is a critical operational concern for agent deployments. Organizations relying on compressed or unstructured memory may face significant performance regressions during model upgrades. Standardized, schema-based memory formats could reduce migration risk and improve long-term agent reliability.
Reducing memory migration risk lowers the total cost of model upgrades and prevents silent performance degradation. Fixed-schema memory can improve agent reliability and reduce engineering effort for maintenance, making it attractive for enterprise AI deployments.
Expect increased adoption of fixed-schema memory architectures for production agents. Further research may quantify portability across larger models and commercial APIs. Tooling for memory migration and embedding version management is likely to emerge as a distinct product category.