SlipSense: Multimodal Tactile Learning for Low-Latency and Generalized Slip Detection
SlipSense is a multimodal tactile slip-detection framework built on TacV5, a compact sensor integrating a 32x32 piezoresistive array operating at 240 Hz and a 3-axis MEMS accelerometer operating at 8 kHz. The framework performs modality-specific encoding, intra-sensor fusion, and cross-modal attention with causal temporal prediction at 240 Hz. Experiments on a dataset of 1.4 million frames spanning 37 objects demonstrate the complementarity of the two modalities. SlipSense achieves 96.7% Macro F1 with a false-positive rate below 1.6%, detecting 76% of slip events within 23.1 ms. When trained solely on UMI data, SlipSense generalizes zero-shot to a Tesollo dexterous hand, transferring across unseen objects, distinct sensor units, and robotic platforms without retraining.
The combination of a high-frequency accelerometer (8 kHz) for vibration cues and a lower-frequency piezoresistive array (240 Hz) for pressure distribution enables complementary slip detection. Causal temporal prediction at 240 Hz ensures low-latency responses. Zero-shot transfer from UMI data to a Tesollo dexterous hand suggests the learned representations are robust across sensor instances and robot platforms.
Low-latency, generalized slip detection is critical for reliable robotic manipulation in unstructured environments. The ability to transfer without retraining reduces deployment costs and accelerates adoption in industrial automation and service robotics.
SlipSense could enable more reliable grasping and manipulation in logistics, manufacturing, and assistive robotics. The zero-shot generalization reduces the need for per-platform data collection, lowering integration costs and time-to-market for robotic solutions.
Next signals include validation on additional robotic hands and real-world manipulation tasks, integration with closed-loop control policies, and potential commercialization of the TacV5 sensor. Further research may explore scaling the dataset and improving robustness to varying surface properties.