MagIK: A Hand-Tracking Magnetic Positioning System Based on a Kinematic Model of the Hand

In this article, we present a hand-tracking system based on magnetic positioning. A single magnetic node is mounted on each fingertip and two magnetic nodes on the back side of the hand. A fixed array of receiving coils is used to detect the magnetic field, from which it is possible to infer the position and orientation of each magnetic node. A kinematic model of the whole hand has been developed. Starting from the positioning data of each magnetic node, the kinematic model can be used to calculate the position and flexion angle of each finger joint, plus the position and orientation of the hand in space. Relying on magnetic fields, the hand-tracking system can work also in non-line-of-sight conditions. The gesture reconstruction is validated by comparing it with a commercial hand-tracking system based on a depth camera. The system requires a small number of electronics to be mounted on the hand. This would allow building a light and comfortable data glove that could be used for several purposes: human–machine interface, sign language recognition, diagnostics, and rehabilitation.

[1]  Michela Borghetti,et al.  Sensor Analysis for a Modular Wearable Finger 3D Motion Tracking System , 2018, Proceedings.

[2]  Lars Bretzner,et al.  Hand gesture recognition using multi-scale colour features, hierarchical models and particle filtering , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[3]  Bibhuti Bhusan Biswal,et al.  Design and Analysis of a Multifingered Robot Hand , 2012, ICRA 2012.

[4]  Jilin Zhou,et al.  A New Hand-Measurement Method to Simplify Calibration in CyberGlove-Based Virtual Rehabilitation , 2010, IEEE Transactions on Instrumentation and Measurement.

[5]  Luigi Ferrigno,et al.  A multi-frequency approach to mitigate the performance degradation of a magnetic positioning system under CW disturbance conditions , 2020 .

[6]  Jun-Sik Kim,et al.  Direct and realistic handover of a virtual object , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[7]  Qiang Li,et al.  A Hand Gesture Recognition Framework and Wearable Gesture-Based Interaction Prototype for Mobile Devices , 2014, IEEE Transactions on Human-Machine Systems.

[8]  John J. Craig,et al.  Introduction to robotics - mechanics and control (2. ed.) , 1989 .

[9]  Pier Paolo Valentini,et al.  Accuracy in fingertip tracking using Leap Motion Controller for interactive virtual applications , 2017 .

[10]  Shana Smith,et al.  A virtual reality keyboard with realistic haptic feedback in a fully immersive virtual environment , 2017, Virtual Reality.

[11]  Tosiyasu L. Kunii,et al.  Constraint-Based Hand Animation , 1993 .

[12]  Michela Borghetti,et al.  Sensorized Glove for Measuring Hand Finger Flexion for Rehabilitation Purposes , 2013, IEEE Transactions on Instrumentation and Measurement.

[13]  Yan Fu,et al.  A Survey on Hand Pose Estimation with Wearable Sensors and Computer-Vision-Based Methods , 2020, Sensors.

[14]  Rahul B. Warrier,et al.  Iterative Learning From Novice Human Demonstrations for Output Tracking , 2016, IEEE Transactions on Human-Machine Systems.

[15]  Alessio De Angelis,et al.  A Multi-Node Magnetic Positioning System with a Distributed Data Acquisition Architecture , 2020, Sensors.

[16]  Mauro Serpelloni,et al.  A Wearable and Wirelessly Powered System for Multiple Finger Tracking , 2020, IEEE Transactions on Instrumentation and Measurement.

[17]  Muhammad Modi bin Lakulu,et al.  A Review on Systems-Based Sensory Gloves for Sign Language Recognition State of the Art between 2007 and 2017 , 2018, Sensors.

[18]  Yunyi Jia,et al.  Controlling Object Hand-Over in Human–Robot Collaboration Via Natural Wearable Sensing , 2019, IEEE Transactions on Human-Machine Systems.

[19]  Mathias Wilhelm,et al.  Whole hand modeling using 8 wearable sensors: biomechanics for hand pose prediction , 2013, AH.

[20]  Antonio Moschitta,et al.  Calibration and Characterization of a Magnetic Positioning System Using a Robotic Arm , 2019, IEEE Transactions on Instrumentation and Measurement.

[21]  Albert Dipanda,et al.  Tracking Hands in Interaction with Objects: A Review , 2017, 2017 13th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS).

[22]  Chin-Shyurng Fahn,et al.  Development of a Fingertip Glove Equipped with Magnetic Tracking Sensors , 2010, Sensors.

[23]  Olivier Gibaru,et al.  Unsupervised Gesture Segmentation by Motion Detection of a Real-Time Data Stream , 2017, IEEE Transactions on Industrial Informatics.

[24]  Junghsi Lee,et al.  Design of an Inertial-Sensor-Based Data Glove for Hand Function Evaluation , 2018, Sensors.

[25]  Giovanni Saggio,et al.  Evaluation of a Stretch Sensor for its inedited application in tracking hand finger movements , 2016, 2016 IEEE International Symposium on Medical Measurements and Applications (MeMeA).

[26]  Andrey Samorodov,et al.  Parkinson’s Disease Detection by Using Machine Learning Algorithms and Hand Movement Signal from LeapMotion Sensor , 2020, 2020 26th Conference of Open Innovations Association (FRUCT).

[27]  Zaid Omar,et al.  A review of hand gesture and sign language recognition techniques , 2017, International Journal of Machine Learning and Cybernetics.

[28]  Anupam Agrawal,et al.  Vision based hand gesture recognition for human computer interaction: a survey , 2012, Artificial Intelligence Review.

[29]  Tosiyasu L. Kunii,et al.  Model-based analysis of hand posture , 1995, IEEE Computer Graphics and Applications.

[30]  I Yavrucuk,et al.  A low cost flight simulator using virtual reality tools , 2008, IEEE Aerospace and Electronic Systems Magazine.

[31]  Alessio De Angelis,et al.  Simultaneous RSS Measurements Using Multiple Inductively Coupled Coils , 2019, 2019 IEEE International Symposium on Measurements & Networking (M&N).

[32]  Khaled Assaleh,et al.  Multiple Proposals for Continuous Arabic Sign Language Recognition , 2019 .

[33]  Kunihiro Chihara,et al.  3D modeling of human hand with motion constraints , 1997, Proceedings. International Conference on Recent Advances in 3-D Digital Imaging and Modeling (Cat. No.97TB100134).

[34]  Saeid Nahavandi,et al.  Control Methods for Internet-Based Teleoperation Systems: A Review , 2019, IEEE Transactions on Human-Machine Systems.

[35]  Albert Dipanda,et al.  Hand pose estimation and tracking in real and virtual interaction: A review , 2019, Image Vis. Comput..

[36]  Michael Girard,et al.  Computer animation of knowledge-based human grasping , 1991, SIGGRAPH.

[37]  Silvia Appendino,et al.  Constraint Study for a Hand Exoskeleton: Human Hand Kinematics and Dynamics , 2013, J. Robotics.

[38]  Luigi Ferrigno,et al.  A real-time tracking system for tremor and trajectory estimation in Parkinson’s disease affected patients , 2020, 2020 IEEE International Symposium on Medical Measurements and Applications (MeMeA).

[39]  Peter H Veltink,et al.  Assessment of hand kinematics using inertial and magnetic sensors , 2014, Journal of NeuroEngineering and Rehabilitation.

[40]  Charles R. Allen,et al.  3-D motion system ("data-gloves"): application for Parkinson's disease , 2003, IEEE Trans. Instrum. Meas..

[41]  Francis L. Merat,et al.  Introduction to robotics: Mechanics and control , 1987, IEEE J. Robotics Autom..

[42]  Kihong Park,et al.  User Study of VR Basic Controller and Data Glove as Hand Gesture Inputs in VR Games , 2017, 2017 International Symposium on Ubiquitous Virtual Reality (ISUVR).

[43]  M. Nowicki,et al.  FOR LEAP MOTION CONTROLLER , 2014 .

[44]  Manuel Ferre,et al.  Efficient human hand kinematics for manipulation tasks , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[45]  Chin-Shyurng Fahn,et al.  Development of a data glove with reducing sensors based on magnetic induction , 2005, IEEE Transactions on Industrial Electronics.

[46]  Andy Cockburn,et al.  User-defined gestures for augmented reality , 2013, INTERACT.

[47]  Nicolas D. Georganas,et al.  Real-Time Hand Gesture Detection and Recognition Using Bag-of-Features and Support Vector Machine Techniques , 2011, IEEE Transactions on Instrumentation and Measurement.

[48]  Changzhi Li,et al.  Noncontact Human–Machine Interface With Planar Probing Coils in a Differential Sensing Architecture , 2018, IEEE Transactions on Instrumentation and Measurement.

[49]  Zheng Wang,et al.  BCL-13: A 13-DOF Soft Robotic Hand for Dexterous Grasping and In-Hand Manipulation , 2018, IEEE Robotics and Automation Letters.

[50]  Yves Rybarczyk,et al.  Implementation and Assessment of an Intelligent Motor Tele-Rehabilitation Platform , 2019, Electronics.

[51]  Yun Pan,et al.  Three-Dimensional Pattern Features in Finger Tapping Test for Patients with Parkinson’s disease , 2020, 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC).

[52]  Thomas M Greiner,et al.  Hand Anthropometry of U.S. Army Personnel , 1991 .

[53]  Alessio De Angelis,et al.  Comparison of Measurement Models for 3D Magnetic Localization and Tracking , 2017, Sensors.

[54]  Leonardo Meli,et al.  GESTO: A Glove for Enhanced Sensing and Touching Based on Inertial and Magnetic Sensors for Hand Tracking and Cutaneous Feedback , 2017, IEEE Transactions on Human-Machine Systems.