A new IMMU-based data glove for hand motion capture with optimized sensor layout

The number of people with hand disabilities caused by stroke is increasing every year. Developing a low-cost and easy-to-use data glove to capture the human hand motion can be used to assess the patient’s hand ability in home environment. While a majority of existing hand motion capture methods are too complex to be used for patients in residential settings. This paper proposes a new sensor layout strategy using the inertial and magnetic measurement units and designs a multi-sensor Kalman data fusion algorithm. The sensor layout strategy is optimized according to the inverse kinematics and the developed hand model, and the number of sensors can be significantly reduced from 12 in conventional systems to 6 in our system with the hand motion being completely and accurately reconstructed. Hand motion capture experiments were conducted on a healthy subject using the developed data glove. The hand motion can be restored completely and the hand gesture can be recognized with an accuracy of 85%. The results of a continuous hand movement indicate an average error under 15% compared with the common glove with full sensors. This new set with optimized sensor layout is promising for lower-cost and residential medical applications.

[1]  Jonas Beskow,et al.  Robust online motion capture labeling of finger markers , 2016, MIG.

[2]  Di Guo,et al.  A novel data glove for fingers motion capture using inertial and magnetic measurement units , 2016, 2016 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[3]  Dana Kulic,et al.  Multi-constrained inverse kinematics for the human hand , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[4]  Jean-Yves Hervé,et al.  Visual hand posture tracking in a gripper guiding application , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[5]  T J Armstrong,et al.  A kinematic model of the human hand to evaluate its prehensile capabilities. , 1992, Journal of biomechanics.

[6]  Zhaojie Ju,et al.  Multimodal Human Hand Motion Sensing and Analysis—A Review , 2019, IEEE Transactions on Cognitive and Developmental Systems.

[7]  Peter H. Veltink,et al.  Measuring 3D Hand and Finger Kinematics—A Comparison between Inertial Sensing and an Opto-Electronic Marker System , 2016, PloS one.

[8]  Martin Saerbeck,et al.  Recent methods and databases in vision-based hand gesture recognition: A review , 2015, Comput. Vis. Image Underst..

[9]  Ping Zhang,et al.  Human–manipulator interface using hybrid sensors with Kalman filters and adaptive multi-space transformation , 2014 .

[10]  Andreas Aristidou,et al.  Hand tracking with physiological constraints , 2016, The Visual Computer.

[11]  Peter H Veltink,et al.  Assessment of hand kinematics using inertial and magnetic sensors , 2014, Journal of NeuroEngineering and Rehabilitation.

[12]  Michael Girard,et al.  Computer animation of knowledge-based human grasping , 1991, SIGGRAPH.

[13]  Thomas Bashford-Rogers,et al.  An evaluation of power transfer functions for HDR video compression , 2018, The Visual Computer.

[14]  K. J. Cole,et al.  Slowing of dexterous manipulation in old age: force and kinematic findings from the ‘nut-and-rod’ task , 2010, Experimental Brain Research.

[15]  Ronan Boulic,et al.  Full-body performance animation with Sequential Inverse Kinematics , 2008, Graph. Model..

[16]  Victor B. Zordan,et al.  Automatic Hand-Over Animation using Principle Component Analysis , 2013, MIG.

[17]  João Carlos Mendes Carvalho,et al.  KINEMATIC MODELING OF A MULTI-FINGERED HAND PROSTHESIS FOR MANIPULATION TASKS , 2015 .

[18]  Chunfang Liu,et al.  3D human gesture capturing and recognition by the IMMU-based data glove , 2018, Neurocomputing.

[19]  A. Geurts,et al.  Definition dependent properties of the cortical silent period in upper-extremity muscles, a methodological study , 2014, Journal of NeuroEngineering and Rehabilitation.

[20]  Po-Tsun Chen,et al.  The repeatability of digital force waveform during natural grasping with five digits , 2016 .

[21]  Osamu Oshiro,et al.  Finger motion capture from wrist-electrode contact resistance , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[22]  Wen-Yuan Chen,et al.  Hand gesture recognition using valley circle feature and Hu’s moments technique for robot movement control , 2016 .

[23]  Helge J. Ritter,et al.  Real-time hand tracking using synergistic inverse kinematics , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[24]  Kang Li,et al.  Development of finger-motion capturing device based on optical linear encoder. , 2011, Journal of rehabilitation research and development.

[25]  Jonas Beskow,et al.  Real-time labeling of non-rigid motion capture marker sets , 2017, Comput. Graph..

[26]  Andreas Aristidou,et al.  Inverse kinematics: a review of existing techniques and introduction of a new fast iterative solver , 2009 .

[27]  Hui Zhao,et al.  Kinematics analysis of a novel all-attitude flight simulator , 2010, Science China Information Sciences.

[28]  Mircea Nicolescu,et al.  Vision-based hand pose estimation: A review , 2007, Comput. Vis. Image Underst..

[29]  P. M. Mendes,et al.  FBG Sensing Glove for Monitoring Hand Posture , 2011, IEEE Sensors Journal.

[30]  Ludovic Hoyet,et al.  Sleight of hand: perception of finger motion from reduced marker sets , 2012, I3D '12.

[31]  Ruize Xu,et al.  MEMS Accelerometer Based Nonspecific-User Hand Gesture Recognition , 2012, IEEE Sensors Journal.

[32]  J. Denavit,et al.  A kinematic notation for lower pair mechanisms based on matrices , 1955 .

[33]  Ahmed Rahmani,et al.  Modeling and Kinematics Study of Hand , 2015, Int. J. Comput. Sci. Appl..

[34]  Robert F. Kirsch,et al.  Miniature Low-Power Inertial Sensors: Promising Technology for Implantable Motion Capture Systems , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[35]  Jonathan Maycock,et al.  Reduced marker layouts for optical motion capture of hands , 2015, MIG.

[36]  W. Phillips,et al.  A Review of Attitude Kinematics for Aircraft Flight Simulation , 2000 .

[37]  Agnès Roby-Brami,et al.  Analysis of grasping strategies and function in hemiparetic patients using an instrumented object , 2013, 2013 IEEE 13th International Conference on Rehabilitation Robotics (ICORR).

[38]  Bryan Buchholz,et al.  Abstract of the XII Congress, International Society of BiomechanicsA kinematic model of the human hand to predict its prehensile capabilities , 1989 .

[39]  Yu Peng,et al.  Development and evaluation of a sensor glove for hand function assessment and preliminary attempts at assessing hand coordination , 2016 .

[40]  M. Latash,et al.  Changes in Finger Coordination and Hand Function with Advanced Age , 2006 .