High-Rate Full-Hand Tactile Sensing for Sim-to-Real Recognition and Grasping
Abstract
Full-hand tactile sensing provides contact information that vision can hardly capture after a dexterous hand encloses an object. This is important for stable grasping and shape recognition under occlusion. This paper presents a full-hand tactile perception framework for sim-to-real dexterous grasping. The platform uses a four-finger dexterous hand with 16 degrees of freedom and an in-house, low-cost piezoresistive tactile sensing system densely integrated across the entire hand, providing stable high-rate tactile signals at 300 Hz. In simulation, MuJoCo is used to match the positions of different tactile sensing taxels. Ray-normal projection and local spacing correction align the tactile layout on the hand surface, while force-voltage calibration maps simulated tactile responses to the real voltage domain. A multi-patch tactile classifier is trained and evaluated with simulated and real data. The model achieves 73.33% accuracy on a real dataset with 150 object-grasp samples, and reaches 92.67% accuracy on the test set after fine-tuning with another 150 real samples. These results show that high-rate full-hand piezoresistive sensing, geometric taxel alignment, and calibrated tactile simulation can support practical grasp-based object shape recognition.
This work develops a high-rate, full-hand piezoresistive tactile perception pipeline for sim-to-real object-shape recognition during dexterous grasping. The system combines a 16-DoF four-finger hand, 12 tactile patches with 620 physical taxels, 300 Hz tactile acquisition, taxel-aligned MuJoCo simulation, and force-voltage calibration.
Zhenyuan Zhang
Dexterous hand design
Qiang Li
Professor and header of AG
My research interests include underwater robot, collaborative robots, humanoid robots.