Human Motion Capturing and Activity Recognition Using Wearable Sensor Networks

Wearable sensor networks enable human motion capture and activity recognition in-field. This technology found widespread use in many areas, where location independent information gathering is useful, e.g., in healthcare and sports, workflow analysis, human-computer-interaction, robotics, and entertainment. Two major approaches for deriving information from wearable sensor networks are in focus here: the model-based estimation of 3D joint kinematics based on networks of inertial measurement units (IMUs) and the activity recognition based on multimodal body sensor networks using machine learning algorithms. The characteristics, working principles, challenges, potentials, and target applications of these two approaches are described individually and in synergy.

[1]  Robert Harle,et al.  A Survey of Indoor Inertial Positioning Systems for Pedestrians , 2013, IEEE Communications Surveys & Tutorials.

[2]  Alison H. McGregor,et al.  Wearable technology for spine movement assessment: A systematic review , 2017, Journal of biomechanics.

[3]  Thomas B. Schön,et al.  An optimization-based approach to human body motion capture using inertial sensors , 2014 .

[4]  Peter H Veltink,et al.  Automatic identification of inertial sensor placement on human body segments during walking , 2013, Journal of NeuroEngineering and Rehabilitation.

[5]  Didier Stricker,et al.  Advanced tracking through efficient image processing and visual-inertial sensor fusion , 2008, 2008 IEEE Virtual Reality Conference.

[6]  Paul Lukowicz,et al.  Sensing muscle activities with body-worn sensors , 2006, International Workshop on Wearable and Implantable Body Sensor Networks (BSN'06).

[7]  Julius Hannink,et al.  Sensor-Based Gait Parameter Extraction With Deep Convolutional Neural Networks , 2016, IEEE Journal of Biomedical and Health Informatics.

[8]  Scott L Delp,et al.  Quantified self and human movement: a review on the clinical impact of wearable sensing and feedback for gait analysis and intervention. , 2014, Gait & posture.

[9]  Patrick Olivier,et al.  A Dynamic Time Warping Approach to Real-Time Activity Recognition for Food Preparation , 2010, AmI.

[10]  Thomas B. Schön,et al.  Using Inertial Sensors for Position and Orientation Estimation , 2017, Found. Trends Signal Process..

[11]  Jun Hu,et al.  Activity recognition based on inertial sensors for Ambient Assisted Living , 2016, 2016 19th International Conference on Information Fusion (FUSION).

[12]  A. Cappozzo,et al.  Human movement analysis using stereophotogrammetry. Part 3. Soft tissue artifact assessment and compensation. , 2005, Gait & posture.

[13]  Guang-Zhong Yang,et al.  Toward Pervasive Gait Analysis With Wearable Sensors: A Systematic Review , 2016, IEEE Journal of Biomedical and Health Informatics.

[14]  Hans-Peter Seidel,et al.  Motion reconstruction using sparse accelerometer data , 2011, TOGS.

[15]  Didier Stricker,et al.  Advanced tracking through efficient image processing and visual-inertial sensor fusion , 2008, VR.

[16]  Thomas Seel,et al.  IMU-Based Joint Angle Measurement for Gait Analysis , 2014, Sensors.

[17]  Gabriele Bleser,et al.  Innovative system for real-time ergonomic feedback in industrial manufacturing. , 2013, Applied ergonomics.

[18]  Paul Lukowicz,et al.  Continuous activity recognition in a maintenance scenario: combining motion sensors and ultrasonic hands tracking , 2012, Pattern Analysis and Applications.

[19]  Peter H. Veltink,et al.  Estimation of Full-Body Poses Using Only Five Inertial Sensors: An Eager or Lazy Learning Approach? , 2016, Sensors.

[20]  Pietro Picerno,et al.  25 years of lower limb joint kinematics by using inertial and magnetic sensors: A review of methodological approaches. , 2017, Gait & posture.

[21]  Paul Lukowicz,et al.  Gesture spotting with body-worn inertial sensors to detect user activities , 2008, Pattern Recognit..

[22]  Felix Wenk,et al.  Posture from motion , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[23]  Bertram Taetz,et al.  IMU-to-Segment Assignment and Orientation Alignment for the Lower Body Using Deep Learning , 2018, Sensors.

[24]  Bertram Taetz,et al.  Real-time inertial lower body kinematics and ground contact estimation at anatomical foot points for agile human locomotion , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[25]  Guang-Zhong Yang,et al.  Wearable Sensing for Solid Biomechanics: A Review , 2015, IEEE Sensors Journal.

[26]  Angelica Munoz-Melendez,et al.  Wearable Inertial Sensors for Human Motion Analysis: A Review , 2016, IEEE Sensors Journal.

[27]  Bertram Taetz,et al.  Towards self-calibrating inertial body motion capture , 2016, 2016 19th International Conference on Information Fusion (FUSION).

[28]  Paul Lukowicz,et al.  Where am I: Recognizing On-body Positions of Wearable Sensors , 2005, LoCA.

[29]  Paul Lukowicz,et al.  Sensor Placement Variations in Wearable Activity Recognition , 2014, IEEE Pervasive Computing.

[30]  Paul Lukowicz,et al.  Using acceleration signatures from everyday activities for on-body device location , 2007, 2007 11th IEEE International Symposium on Wearable Computers.

[31]  Kamiar Aminian,et al.  Validation of functional calibration and strap-down joint drift correction for computing 3D joint angles of knee, hip, and trunk in alpine skiing , 2017, PloS one.

[32]  Jingyuan Cheng,et al.  A simple wristband based on capacitive sensors for recognition of complex hand motions , 2012, 2012 IEEE Sensors.

[33]  Gernot Bahle,et al.  On the Use of Magnetic Field Disturbances as Features for Activity Recognition with on Body Sensors , 2010, EuroSSC.

[34]  Brice Bouvier,et al.  Upper Limb Kinematics Using Inertial and Magnetic Sensors: Comparison of Sensor-to-Segment Calibrations , 2015, Sensors.

[35]  A. Leardini,et al.  A new anatomically based protocol for gait analysis in children. , 2007, Gait & posture.

[36]  Ivan Tashev,et al.  Unsupervised deep representation learning to remove motion artifacts in free-mode body sensor networks , 2017, 2017 IEEE 14th International Conference on Wearable and Implantable Body Sensor Networks (BSN).

[37]  Michael Schwenk,et al.  Effects of Wearable Sensor-Based Balance and Gait Training on Balance, Gait, and Functional Performance in Healthy and Patient Populations: A Systematic Review and Meta-Analysis of Randomized Controlled Trials , 2017, Gerontology.

[38]  Fredrik Olsson,et al.  Experimental evaluation of joint position estimation using inertial sensors , 2017, 2017 20th International Conference on Information Fusion (Fusion).

[39]  Paul Lukowicz,et al.  Analysis of Chewing Sounds for Dietary Monitoring , 2005, UbiComp.

[40]  Thomas Seel,et al.  Automatic pairing of inertial sensors to lower limb segments – a plug-and-play approach , 2016 .

[41]  Aurelio Cappozzo,et al.  A hip joint kinematics driven model for the generation of realistic thigh soft tissue artefacts. , 2013, Journal of biomechanics.

[42]  Marcelo Bissi Pires,et al.  Sensor Fusion and Smart Sensor in Sports and Biomedical Applications , 2016, Sensors.

[43]  Dima Damen,et al.  Cognitive Learning, Monitoring and Assistance of Industrial Workflows Using Egocentric Sensor Networks , 2015, PloS one.

[44]  Angelo M. Sabatini,et al.  Dealing with Magnetic Disturbances in Human Motion Capture: A Survey of Techniques , 2016, Micromachines.

[45]  Paul Lukowicz,et al.  Activity Recognition of Assembly Tasks Using Body-Worn Microphones and Accelerometers , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Paul Lukowicz,et al.  Using FSR based muscule activity monitoring to recognize manipulative arm gestures , 2007, 2007 11th IEEE International Symposium on Wearable Computers.

[47]  Jeroen D. Hol,et al.  Sensor Fusion and Calibration of Inertial Sensors, Vision, Ultra-Wideband and GPS , 2011 .

[48]  D. Sutherland The evolution of clinical gait analysis. Part II kinematics. , 2002, Gait & posture.

[49]  Daniel Steffen,et al.  jumpBALL - Ein mobiles Exergame für die Thromboseprophylaxe , 2017, Mensch & Computer.

[50]  Jasper Reenalda,et al.  Continuous three dimensional analysis of running mechanics during a marathon by means of inertial magnetic measurement units to objectify changes in running mechanics. , 2016, Journal of biomechanics.

[51]  Paul Lukowicz,et al.  In the blink of an eye: combining head motion and eye blink frequency for activity recognition with Google Glass , 2014, AH.

[52]  Bertram Taetz,et al.  Development of an Inertial Motion Capture System for Clinical Application , 2017, i-com.

[53]  Carmen C. Y. Poon,et al.  Unobtrusive Sensing and Wearable Devices for Health Informatics , 2014, IEEE Transactions on Biomedical Engineering.

[54]  Dai Meng,et al.  Accuracy Improvement on the Measurement of Human-Joint Angles , 2016, IEEE Journal of Biomedical and Health Informatics.

[55]  Viral H. Borisagar,et al.  A Literature Survey on Human Activity Recognition via Hidden Markov Model , 2013 .

[56]  Markus Weber,et al.  Personalized Physical Activity Monitoring Using Wearable Sensors , 2015, Smart Health.

[57]  Ling Bao,et al.  Activity Recognition from User-Annotated Acceleration Data , 2004, Pervasive.

[58]  Nevio Luigi Tagliamonte,et al.  Assessment of lower limbs kinematics during human–robot interaction using inertial measurement units , 2014 .

[59]  Bodo Rosenhahn,et al.  Sparse Inertial Poser: Automatic 3D Human Pose Estimation from Sparse IMUs , 2017, Comput. Graph. Forum.

[60]  Leonidas J. Guibas,et al.  Shape google: Geometric words and expressions for invariant shape retrieval , 2011, TOGS.

[61]  Yeti Li,et al.  Automated Rehabilitation System: Movement Measurement and Feedback for Patients and Physiotherapists in the Rehabilitation Clinic , 2016, Hum. Comput. Interact..

[62]  Daniel Roggen,et al.  Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition , 2016, Sensors.

[63]  Gustaf Hendeby,et al.  A personalized exercise trainer for elderly , 2011, 2011 5th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops.

[64]  Stefano Paolucci,et al.  Wearable inertial sensors for human movement analysis , 2016, Expert review of medical devices.

[65]  Thomas B. Schön,et al.  Indoor Positioning Using Ultrawideband and Inertial Measurements , 2015, IEEE Transactions on Vehicular Technology.

[66]  Albrecht Schmidt,et al.  Advanced Interaction in Context , 1999, HUC.

[67]  Bertram Taetz,et al.  On Inertial Body Tracking in the Presence of Model Calibration Errors , 2016, Sensors.