MyoMechanix: Biomechanically-Grounded Compositional Skilled Activity Understanding and Coaching
MyoMechanix is a multimodal ecosystem for weight-loaded actions that aligns motion with muscle activity. It contains 7,500+ samples of 20 actions from 38 subjects, with synchronized multiview RGB video, 3D pose, sEMG, and additional physiological signals, forming the largest multimodal AQA benchmark to date. The Fitness Knowledge Graph (FKG) organizes expert annotations into structured relationships among actions, phases, key steps, errors, and corrective feedback. CUBIST (Compositional Ontological Reasoning Engine) performs decomposition-analysis-recomposition for fine-grained error attribution and feedback generation. The work establishes MyoMechanix-AQA, MyoMechanix-VideoQA, and MyoMechanix-Video2EMG tasks.
Researchers introduce MyoMechanix, a multimodal dataset and benchmark for biomechanically grounded action quality assessment in weight-loaded exercises. It includes synchronized video, pose, sEMG, and physiological data from 38 subjects performing 20 actions, with expert annotations structured into a Fitness Knowledge Graph. The CUBIST reasoning engine uses compositional analysis to provide fine-grained error attribution and coaching feedback. The paper also defines three tasks: MyoMechanix-AQA, MyoMechanix-VideoQA, and MyoMechanix-Video2EMG.
The integration of sEMG with visual and pose data enables models to learn muscle-level dynamics that are invisible in RGB-only approaches. The Fitness Knowledge Graph allows compositional scoring by decomposing actions into phases and key steps, potentially improving generalization to unseen action combinations. The Video2EMG task suggests a pathway to predict muscle activation from video alone, which could reduce sensor requirements in real-world deployment.
This work targets the fitness and rehabilitation coaching market, where automated, biomechanically accurate feedback could differentiate products. The dataset's scale and multimodal nature may attract commercial interest from wearable and computer vision companies. The structured knowledge graph approach could be adapted to other skilled physical activities, such as physical therapy or sports training.
The benchmark and reasoning engine could underpin AI-powered personal training and rehabilitation products, offering fine-grained, explainable feedback. The multimodal dataset lowers the barrier for startups to develop biomechanically aware coaching systems. The knowledge graph structure supports scalable, interpretable assessment across many exercises, potentially reducing expert annotation costs.
Next observable signals include follow-up papers applying CUBIST to other datasets, release of code and pretrained models, and potential industry partnerships for fitness coaching applications. The Video2EMG task may spur research into cross-modal prediction and reduce reliance on wearable sensors. Adoption in consumer fitness apps or clinical rehabilitation tools would indicate commercial validation.