InstructMesh: Selective Refinement of Generative 3D Models for Fabrication
InstructMesh is an interactive post-generation refinement tool that enables selective repair of generative 3D models through region selection and targeted operations, such as opening or sealing voids, or adjusting local thickness. Users can invoke edit operations via natural language prompts or slider controls. The tool operates directly on the intermediate latent representation, allowing robust geometric corrections without expert modeling skills. The research includes analysis of common fabrication-related failure modes in outputs from state-of-the-art generative tools and two user studies demonstrating that novices can identify and perform fabrication-relevant repairs using InstructMesh, with a preference for hybrid interfaces combining slider controls and natural language input.
InstructMesh is an interactive post-generation refinement tool for generative 3D models, enabling selective repair through region selection and targeted operations like opening/sealing voids or adjusting local thickness. Users can invoke edits via natural language prompts or slider controls. It operates on the intermediate latent representation, allowing robust geometric corrections without expert modeling skills. The research analyzes common fabrication-related failure modes in state-of-the-art generative outputs and includes two user studies showing novices can perform fabrication-relevant repairs, with a preference for hybrid interfaces combining sliders and natural language.
InstructMesh operates directly on the intermediate latent representation of generative 3D models, enabling selective geometric corrections without requiring expert modeling skills. The tool supports region selection and targeted operations such as opening or sealing voids and adjusting local thickness. Users can invoke edits via natural language prompts or slider controls. The research includes analysis of common fabrication-related failure modes in outputs from state-of-the-art generative tools.
The tool addresses a gap between generative 3D model creation and fabrication readiness, where visual plausibility often outweighs geometric accuracy. By enabling novices to perform fabrication-relevant repairs, InstructMesh could lower the barrier to using generative AI for physical production. The preference for hybrid interfaces combining sliders and natural language suggests a design direction for future creative tools.
InstructMesh could reduce the need for expert 3D modeling skills in fabrication pipelines, potentially lowering costs and accelerating iteration for prototyping and manufacturing. It may enable broader use of generative 3D models in industries requiring physical outputs.
Potential next signals include adoption of InstructMesh or similar tools in 3D printing workflows, integration with commercial generative 3D platforms, and further research on automated detection of fabrication flaws. The hybrid interface approach may influence other generative design tools.