A wearable motion capture device able to detect dynamic motion of human limbs

Limb motion capture is essential in human motion-recognition, motor-function assessment and dexterous human-robot interaction for assistive robots. Due to highly dynamic nature of limb activities, conventional inertial methods of limb motion capture suffer from serious drift and instability problems. Here, a motion capture method with integral-free velocity detection is proposed and a wearable device is developed by incorporating micro tri-axis flow sensors with micro tri-axis inertial sensors. The device allows accurate measurement of three-dimensional motion velocity, acceleration, and attitude angle of human limbs in daily activities, strenuous, and prolonged exercises. Additionally, we verify an intra-limb coordination relationship exists between thigh and shank in human walking and running, and establish a neural network model for it. Using the intra-limb coordination model, dynamic motion capture of human lower limbs including thigh and shank is tactfully implemented by a single shank-worn device, which simplifies the capture device and reduces cost. Experiments in strenuous activities and long-time running validate excellent performance and robustness of the wearable device in dynamic motion recognition and reconstruction of human limbs.

[1]  M Rahul,et al.  Review on Motion Capture Technology , 2018 .

[2]  Peng Liu,et al.  A Flexible Flow Sensor System and Its Characteristics for Fluid Mechanics Measurements , 2009, Sensors.

[3]  Ignacio Galiana,et al.  Reducing the metabolic rate of walking and running with a versatile, portable exosuit , 2019, Science.

[4]  Jinxian Yang,et al.  A Novel Orientation Recursive Algorithm Aiming at Catastrophe Signals in MWD , 2020, IEEE Transactions on Industrial Electronics.

[5]  William H Gage,et al.  Kinematic and kinetic validity of the inverted pendulum model in quiet standing. , 2004, Gait & posture.

[6]  Negar Golestani,et al.  Human activity recognition using magnetic induction-based motion signals and deep recurrent neural networks , 2020, Nature Communications.

[7]  Gongfa Li,et al.  A novel feature extraction method for machine learning based on surface electromyography from healthy brain , 2019, Neural Computing and Applications.

[8]  Jaime E. Duarte,et al.  The Myosuit: Bi-articular Anti-gravity Exosuit That Reduces Hip Extensor Activity in Sitting Transfers , 2017, Front. Neurorobot..

[9]  F. Lacquaniti,et al.  Kinematic coordination in human gait: relation to mechanical energy cost. , 1998, Journal of neurophysiology.

[10]  Joao Ramos,et al.  Dynamic Bilateral Teleoperation of the Cart-Pole: A Study Toward the Synchronization of Human Operator and Legged Robot , 2018, IEEE Robotics and Automation Letters.

[11]  Mathew Yarossi,et al.  Changes in center of pressure displacement with the use of a foot drop stimulator in individuals with stroke. , 2015, Clinical biomechanics.

[12]  Tim J Gabbett,et al.  The Use of Wearable Microsensors to Quantify Sport-Specific Movements , 2015, Sports Medicine.

[13]  Sheryl Coombs,et al.  Smart Skins: Information Processing by Lateral Line Flow Sensors , 2001, Auton. Robots.

[14]  R. Kram,et al.  The effects of adding mass to the legs on the energetics and biomechanics of walking. , 2007, Medicine and science in sports and exercise.

[15]  Rong Zhu,et al.  A Compact Flexible Thermal Flow Sensor for Detecting Two-Dimensional Flow Vector , 2015, IEEE Sensors Journal.

[16]  Angelo M. Sabatini,et al.  A Novel Kalman Filter for Human Motion Tracking With an Inertial-Based Dynamic Inclinometer , 2015, IEEE Transactions on Biomedical Engineering.

[17]  Franklin Bien,et al.  Wearable smart sensor systems integrated on soft contact lenses for wireless ocular diagnostics , 2017, Nature Communications.

[18]  Jayoung Kim,et al.  Wearable biosensors for healthcare monitoring , 2019, Nature Biotechnology.

[19]  Jung-Keun Lee,et al.  Estimation of Attitude and External Acceleration Using Inertial Sensor Measurement During Various Dynamic Conditions , 2012, IEEE Transactions on Instrumentation and Measurement.

[20]  Yong Yan,et al.  Quantitative Assessment of Upper Limb Motion in Neurorehabilitation Utilizing Inertial Sensors , 2015, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[21]  Rong Zhu,et al.  Motion Velocity, Acceleration and Energy Expenditure Estimation Using Micro Flow Sensor , 2019, IEEE Access.

[22]  Francisco B. Ortega,et al.  Accelerometer Data Collection and Processing Criteria to Assess Physical Activity and Other Outcomes: A Systematic Review and Practical Considerations , 2017, Sports Medicine.

[23]  Sangbae Kim,et al.  Dynamic locomotion synchronization of bipedal robot and human operator via bilateral feedback teleoperation , 2019, Science Robotics.

[24]  T. Flash,et al.  An analytical formulation of the law of intersegmental coordination during human locomotion , 2009, Experimental Brain Research.

[25]  H J Stam,et al.  Validity of the Pedar Mobile system for vertical force measurement during a seven-hour period. , 2006, Journal of biomechanics.

[26]  Shi Qiang Liu,et al.  A Wearable Flow-MIMU Device for Monitoring Human Dynamic Motion , 2020, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[27]  King H. Yang,et al.  Relationship between lower limb dynamics and knee joint pain , 1991, Journal of orthopaedic research : official publication of the Orthopaedic Research Society.

[28]  Honghai Liu,et al.  Surface EMG data aggregation processing for intelligent prosthetic action recognition , 2018, Neural Computing and Applications.

[29]  Qiang Yang,et al.  Sensor-Based Abnormal Human-Activity Detection , 2008, IEEE Transactions on Knowledge and Data Engineering.

[30]  Robert J. Wood,et al.  Wearable soft sensing suit for human gait measurement , 2014, Int. J. Robotics Res..

[31]  Jasper Reenalda,et al.  Continuous three dimensional analysis of running mechanics during a marathon by means of inertial magnetic measurement units to objectify changes in running mechanics. , 2016, Journal of biomechanics.

[32]  R. Zhu,et al.  Temperature and Strain Compensation for Flexible Sensors based on Thermosensation. , 2019, ACS applied materials & interfaces.

[33]  Zhaoying Zhou,et al.  A real-time articulated human motion tracking using tri-axis inertial/magnetic sensors package. , 2004, IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[34]  Ya Tian,et al.  An Adaptive-Gain Complementary Filter for Real-Time Human Motion Tracking With MARG Sensors in Free-Living Environments , 2013, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[35]  Shi Qiang Liu,et al.  A Wearable Human Motion Tracking Device Using Micro Flow Sensor Incorporating a Micro Accelerometer , 2020, IEEE Transactions on Biomedical Engineering.

[36]  Conor J Walsh,et al.  Wearable Movement Sensors for Rehabilitation: A Focused Review of Technological and Clinical Advances , 2018, PM & R : the journal of injury, function, and rehabilitation.

[37]  Guido Pasquini,et al.  Aging does not affect the intralimb coordination elicited by slip-like perturbation of different intensities. , 2017, Journal of neurophysiology.

[38]  Hyun Jin Kim,et al.  A Human Motion Tracking Algorithm Using Adaptive EKF Based on Markov Chain , 2016, IEEE Sensors Journal.

[39]  Mahmoud El-Gohary,et al.  Human Joint Angle Estimation with Inertial Sensors and Validation with A Robot Arm , 2015, IEEE Transactions on Biomedical Engineering.

[40]  Scott Kuindersma,et al.  Human-in-the-loop optimization of hip assistance with a soft exosuit during walking , 2018, Science Robotics.

[41]  Krzysztof Brzostowski Novel approach to human walking speed enhancement based on drift estimation , 2018, Biomed. Signal Process. Control..

[42]  Joonho Lee,et al.  Learning agile and dynamic motor skills for legged robots , 2019, Science Robotics.

[43]  Subhas Chandra Mukhopadhyay,et al.  Wearable Sensors for Human Activity Monitoring: A Review , 2015, IEEE Sensors Journal.

[44]  Dongni Johansson,et al.  Wearable sensors for clinical applications in epilepsy, Parkinson’s disease, and stroke: a mixed-methods systematic review , 2018, Journal of Neurology.

[45]  Conor J. Walsh,et al.  A Highly Sensitive Capacitive‐Based Soft Pressure Sensor Based on a Conductive Fabric and a Microporous Dielectric Layer , 2018 .

[46]  Rong Zhu,et al.  Wearable Anemometer With Multi-Sensing of Wind Absolute Orientation, Wind Speed, Attitude, and Heading , 2019, IEEE Sensors Journal.

[47]  Wei Tang,et al.  Self ‐Powered Insole Plantar Pressure Mapping System , 2018, Advanced Functional Materials.

[48]  Meena Kumari,et al.  Genetic studies of accelerometer-based sleep measures yield new insights into human sleep behaviour , 2019, Nature Communications.

[49]  Daniel M. Vogt,et al.  Batch Fabrication of Customizable Silicone‐Textile Composite Capacitive Strain Sensors for Human Motion Tracking , 2017 .

[50]  Shi Qiang Liu,et al.  A Complementary Filter Based on Multi-Sample Rotation Vector for Attitude Estimation , 2018, IEEE Sensors Journal.

[51]  Alin Albu-Schäffer,et al.  Three-Dimensional Bipedal Walking Control Based on Divergent Component of Motion , 2015, IEEE Transactions on Robotics.

[52]  Jae-Woong Jeong,et al.  Epidermal electronics for electromyography: An application to swallowing therapy. , 2016, Medical engineering & physics.

[53]  Wen-An Zhang,et al.  A Novel Adaptive Kalman Filtering Approach to Human Motion Tracking With Magnetic-Inertial Sensors , 2020, IEEE Transactions on Industrial Electronics.

[54]  Xiaoli Meng,et al.  Self-Contained Pedestrian Tracking During Normal Walking Using an Inertial/Magnetic Sensor Module , 2014, IEEE Transactions on Biomedical Engineering.

[55]  Rong Zhu,et al.  A Two-Dimensional Flow Sensor with Integrated Micro Thermal Sensing Elements and a Back Propagation Neural Network , 2013, Sensors.

[56]  Andrea Mannini,et al.  Walking speed estimation using foot-mounted inertial sensors: comparing machine learning and strap-down integration methods. , 2014, Medical engineering & physics.

[57]  Gerhard Tröster,et al.  Monitoring Kinematic Changes With Fatigue in Running Using Body-Worn Sensors , 2012, IEEE Transactions on Information Technology in Biomedicine.

[58]  Duli Yu,et al.  Flexible and Stretchable Electronic Skin with High Durability and Shock Resistance via Embedded 3D Printing Technology for Human Activity Monitoring and Personal Healthcare , 2019, Advanced Materials Technologies.

[59]  N. A. Borghese,et al.  Kinematic determinants of human locomotion. , 1996, The Journal of physiology.

[60]  I-Ming Chen,et al.  Localization and velocity tracking of human via 3 IMU sensors , 2014 .

[61]  Q Li,et al.  Walking speed estimation using a shank-mounted inertial measurement unit. , 2010, Journal of biomechanics.