Reconstructing Human Hand Pose and Configuration using a Fixed-Base Exoskeleton

Accurate real-time estimation of the pose and configuration of the human hand attached to a dexterous haptic input device is crucial to improve the interaction possibilities for teleoperation and in virtual and augmented reality. In this paper, we present an approach to reconstruct the pose of the human hand and the joint angles of the fingers when wearing a novel fixed-base (grounded) hand exoskeleton. Using a kinematic model of the human hand built from MRI data, we can reconstruct the hand pose and joint angles without sensors on the human hand, from attachment points on the first three fingers and the palm. We test the accuracy of our approach using motion capture as a ground truth. This reconstruction can be used to determine contact geometry and force-feedback from virtual or remote objects in virtual reality or teleoperation.

[1]  Wael Bachta,et al.  Analysis of the directions in which forces are applied on the hand during manual manipulation and exploration , 2015, 2015 IEEE World Haptics Conference (WHC).

[2]  Manuel Ferre,et al.  Efficient human hand kinematics for manipulation tasks , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Kaspar Althoefer,et al.  The Role of the Thumb: Study of Finger Motion in Grasping and Reachability Space in Human and Robotic Hands , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[4]  P. van der Smagt,et al.  Generating marker stars for 6D optical tracking , 2012, 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[5]  Hong Liu,et al.  Multisensory five-finger dexterous hand: The DLR/HIT Hand II , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[6]  Haruhisa Kawasaki,et al.  Five-Fingered Haptic Interface Robot: HIRO III , 2009, IEEE Transactions on Haptics.

[7]  Haruhisa Kawasaki,et al.  Collision avoidance and its experimental investigation for a side-faced-type multi-fingered haptic interface , 2014, 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[8]  Koichi Hirota,et al.  Interaction with virtual object using deformable hand , 2016, 2016 IEEE Virtual Reality (VR).

[9]  Michael Girard,et al.  Computer animation of knowledge-based human grasping , 1991, SIGGRAPH.

[10]  Thomas Hulin,et al.  Time Domain Passivity Control for multi-degree of freedom haptic devices with time delay , 2010, 2010 IEEE International Conference on Robotics and Automation.

[11]  W. Wolovich,et al.  A computational technique for inverse kinematics , 1984, The 23rd IEEE Conference on Decision and Control.

[12]  Zhaopeng Chen,et al.  Toward a task space framework for gesture commanded telemanipulation , 2012, 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication.

[13]  Alexander Dietrich,et al.  An overview of null space projections for redundant, torque-controlled robots , 2015, Int. J. Robotics Res..

[14]  Leonardo Meli,et al.  GESTO: A Glove for Enhanced Sensing and Touching Based on Inertial and Magnetic Sensors for Hand Tracking and Cutaneous Feedback , 2017, IEEE Transactions on Human-Machine Systems.

[15]  Lorenzo Sciavicco,et al.  Robust Control of Robotic Manipulators , 1984 .

[16]  Sadao Kawamura,et al.  Analysis of friction on human fingers and design of artificial fingers , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[17]  Thomas Hulin,et al.  The STAMAS Simulator: A Kinematics and Dynamics Simulator for an Astronaut's Leg and Hand Exoskeleton , 2015 .

[18]  Michela Borghetti,et al.  Sensorized Glove for Measuring Hand Finger Flexion for Rehabilitation Purposes , 2013, IEEE Transactions on Instrumentation and Measurement.

[19]  Neff Walker,et al.  Evaluation of the CyberGlove as a whole-hand input device , 1995, TCHI.

[20]  Leonardo Meli,et al.  Multicontact Bilateral Telemanipulation With Kinematic Asymmetries , 2017, IEEE/ASME Transactions on Mechatronics.

[21]  Christophe Chaillou,et al.  Mechanical Responses of the Fingerpad and Distal Phalanx to Friction of a Grooved Surface: Effect of the Contact Angle , 2006, 2006 14th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[22]  Alessandro De Luca,et al.  Friction observer and compensation for control of robots with joint torque measurement , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[23]  Antonio Frisoli,et al.  A new force-feedback arm exoskeleton for haptic interaction in virtual environments , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[24]  Patrick van der Smagt,et al.  MRI-Based Skeletal Hand Movement Model , 2014, The Human Hand as an Inspiration for Robot Hand Development.