A Robust Data-Driven Soft Sensory Glove for Human Hand Motions Identification and Replication

Biomechanical sensors are essential components for wearable robots because of their broad spectrum of industrial and medical applications. A wide range of these sensors uses soft materials with deformation-related electrical properties, namely variable resistors and capacitors. Previously, conductive materials were injected into silicone structures that require several molding steps to build the microchannels. This paper proposes a versatile soft sensing glove, using a simple process for preparing sensors of different sizes without molding. We used commercially available silicone tubes to host the conductive liquid. Ten sensors were attached to the back side of the glove to measure flexion-extension and four sensors were placed on the glove at the interdigital folds between the fingers to measure abduction-adduction. The sensory glove successfully replicates hand motion. We used machine learning algorithms to estimate the angles of the joints in the hand and also to identify 15 gestures. The system’s robustness was evaluated in two experiments. The gesture prediction is robust to shocks from contact with a punching ball and also submersion of the glove in water. The proposed sensory glove overcomes challenges of comfort, rigidity, and robustness. It can be used, therefore, to replicate human hand motion in industrial and medical applications.

[1]  Lei Wang,et al.  Microchannel Structural Design For a Room-Temperature Liquid Metal Based Super-stretchable Sensor , 2019, Scientific Reports.

[2]  Michael Neff,et al.  State of the Art in Hand and Finger Modeling and Animation , 2015, Comput. Graph. Forum.

[3]  Robert J. Wood,et al.  Biocompatible Soft Fluidic Strain and Force Sensors for Wearable Devices , 2018, Advanced functional materials.

[4]  Joan Condell,et al.  IMU Sensor-Based Electronic Goniometric Glove for Clinical Finger Movement Analysis , 2018, IEEE Sensors Journal.

[5]  Xiaomin Ren,et al.  Tracing the Motion of Finger Joints for Gesture Recognition via Sewing RGO-Coated Fibers Onto a Textile Glove , 2019, IEEE Sensors Journal.

[6]  Chwee Teck Lim,et al.  Tactile sensorized glove for force and motion sensing , 2016, 2016 IEEE SENSORS.

[7]  Fred H. Hamker,et al.  Tactile and Proximity Servoing by a Multi-modal Sensory Soft Hand , 2018 .

[8]  Yiwei Tao,et al.  Wearable soft artificial skin for hand motion detection with embedded microfluidic strain sensing , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[9]  DaeEun Kim,et al.  Finger-gesture Recognition Glove using Velostat (ICCAS 2011) , 2011, 2011 11th International Conference on Control, Automation and Systems.

[10]  Junghsi Lee,et al.  Design of an Inertial-Sensor-Based Data Glove for Hand Function Evaluation , 2018, Sensors.

[11]  Suin Kim,et al.  A Novel All-in-One Manufacturing Process for a Soft Sensor System and its Application to a Soft Sensing Glove , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[12]  Constantin F. Aliferis,et al.  A comprehensive comparison of random forests and support vector machines for microarray-based cancer classification , 2008, BMC Bioinformatics.

[13]  Ling Li,et al.  SkinGest: artificial skin for gesture recognition via filmy stretchable strain sensors* , 2018, Adv. Robotics.

[14]  Suin Kim,et al.  Development of a wearable soft sensor system for measuring finger motions , 2016, 2016 16th International Conference on Control, Automation and Systems (ICCAS).

[15]  Suin Kim,et al.  A Soft Sensor-Based Three-Dimensional (3-D) Finger Motion Measurement System , 2017, Sensors.

[16]  Masahiko Toyonaga,et al.  Data-Glove for Japanese Sign Language Training System with Gyro-Sensor , 2018, 2018 Joint 10th International Conference on Soft Computing and Intelligent Systems (SCIS) and 19th International Symposium on Advanced Intelligent Systems (ISIS).

[17]  John Nassour,et al.  Design of new Sensory Soft Hand: Combining air-pump actuation with superimposed curvature and pressure sensors , 2018, 2018 IEEE International Conference on Soft Robotics (RoboSoft).

[18]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[19]  Dario Floreano,et al.  Ultrastretchable Strain Sensors Using Carbon Black‐Filled Elastomer Composites and Comparison of Capacitive Versus Resistive Sensors , 2018 .

[20]  Gordon Cheng,et al.  Robust Tactile Descriptors for Discriminating Objects From Textural Properties via Artificial Robotic Skin , 2018, IEEE Transactions on Robotics.

[21]  Yun Feng Fu,et al.  Biomechanical Modeling of Human Body Movement , 2016 .

[22]  Buntarou Shizuki,et al.  A Sensing Technique for Data Glove Using Conductive Fiber , 2019, CHI Extended Abstracts.

[23]  Gordon Cheng,et al.  Active Tactile Transfer Learning for Object Discrimination in an Unstructured Environment Using Multimodal Robotic Skin , 2017, Int. J. Humanoid Robotics.

[24]  Gregory de Boer,et al.  Robust and high-performance soft inductive tactile sensors based on the Eddy-current effect , 2018 .

[25]  Joo Chuan Yeo,et al.  Flexible and Stretchable Strain Sensing Actuator for Wearable Soft Robotic Applications , 2016 .

[26]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[27]  Robert J. Wood,et al.  Toward a modular soft sensor-embedded glove for human hand motion and tactile pressure measurement , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[28]  P. Culmer,et al.  Design and Characterization of Tri-Axis Soft Inductive Tactile Sensors , 2018, IEEE Sensors Journal.