GraFT: A Training-Free Framework for Spatial Reasoning in Multimodal Large Language Models via 3D Scene Graphs
GraFT is a training-free framework that supplies missing 3D structure through a compact 3D scene graph (3DSG). It provides three spatial reasoning capabilities: deterministic geometry through symbolic tools, allocentric layout through a bird's-eye-view rendering, and visual-attribute grounding through task-relevant egocentric frames. On ScanQA, GraFT improves every metric over the same-backbone baseline, raising CIDEr by 27%. On VSI-Bench, GraFT improves frozen MLLMs by up to 65%, surpassing every proprietary and general-purpose open-source baseline.
GraFT introduces a training-free approach to enhance spatial reasoning in multimodal large language models by leveraging 3D scene graphs. It avoids costly fine-tuning and dedicated 3D encoders, instead using symbolic tools, BEV renderings, and egocentric frames to improve geometric measurement, viewpoint transformation, and visual grounding. Reported gains include a 27% CIDEr improvement on ScanQA and up to 65% improvement on VSI-Bench over frozen MLLMs.
The framework decouples spatial reasoning from model weights by injecting structured 3D information at inference time. Key technical signals to watch include whether the 3DSG construction is automated or manual, the computational overhead of BEV rendering and symbolic tool calls, and generalization to other MLLM backbones beyond the tested baseline.
Training-free methods reduce the barrier to improving spatial reasoning in deployed MLLMs, potentially accelerating adoption in robotics, AR/VR, and embodied AI without retraining costs. The reported gains over proprietary baselines suggest a competitive open-source path for spatial intelligence.
By avoiding fine-tuning and dedicated encoders, GraFT could lower the cost and complexity of adding spatial reasoning to commercial MLLM products, enabling faster feature deployment and differentiation in spatial AI applications.
Next observable signals include peer-reviewed validation, release of code and 3DSG datasets, and independent benchmarks on additional spatial reasoning tasks. If the approach scales, it may influence model providers to adopt scene-graph-based inference pipelines.