Token-Efficient Data Reasoning Agents via Adaptive Structuring of Unstructured Data
A research paper proposes agentic data cracking, a method that structures unstructured data adaptively and speculatively as a byproduct of reasoning. On the FanOutQA benchmark, reasoning over an ideal pre-structured store is 28X cheaper than over unstructured data, with the gap growing as questions fan out over more documents.
The paper addresses the high token cost of LLM agents reasoning over unstructured data. It introduces agentic data cracking, which structures data adaptively based on observed queries and speculatively beyond the current question. The method aims to reduce cost by turning repeated document opening into cheaper structured lookups.
The approach leverages query-driven adaptive structuring, meaning the system decides when and what to structure based on incoming queries, and performs speculative structuring beyond the immediate question. This contrasts with pre-structuring all data, which is infeasible due to the vast possible structures and unknown future queries.
Enterprise AI deployments using LLM agents over unstructured data face prohibitive token costs. This method could enable more cost-effective reasoning agents, potentially accelerating adoption in knowledge-worker applications where data is scattered across documents.
Reducing token consumption by up to 28X or more for multi-document reasoning can significantly lower operational costs for AI-powered enterprise search and analysis tools, improving ROI and scalability.
Next signals include empirical validation on additional benchmarks beyond FanOutQA, integration into agent frameworks, and commercial pilots measuring cost savings in real enterprise workloads.