Intention Prediction and Human Health Condition Detection in Reaching Tasks with Machine Learning Techniques

Detecting human motion and predicting human intentions by analyzing body signals are challenging but fundamental steps for the implementation of applications presenting human–robot interaction in different contexts, such as robotic rehabilitation in clinical environments, or collaborative robots in industrial fields. Machine learning techniques (MLT) can face the limit of small data amounts, typical of this kind of applications. This paper studies the illustrative case of the reaching movement in 10 healthy subjects and 21 post-stroke patients, comparing the performance of linear discriminant analysis (LDA) and random forest (RF) in: (i) predicting the subject’s intention of moving towards a specific direction among a set of possible choices, (ii) detecting if the subject is moving according to a healthy or pathological pattern, and in the case of discriminating the damage location (left or right hemisphere). Data were captured with wearable electromagnetic sensors, and a sub-section of the acquired signals was required for the analyses. The possibility of detecting with which arm (left or right hand) the motion was performed, and the sensitivity of the MLT to variations in the length of the signal sub-section were also evaluated. LDA and RF prediction accuracies were compared: Accuracy improves when only healthy subjects or longer signals portions are considered up to 11% and at least 10%, respectively. RF reveals better estimation performance both as intention predictor (on average 59.91% versus the 62.19% of LDA), and health condition detector (over 90% in all the tests).

[1]  Geng Yang,et al.  Finger Angle-Based Hand Gesture Recognition for Smart Infrastructure Using Wearable Wrist-Worn Camera , 2018 .

[2]  Ludovic Saint-Bauzel,et al.  Inclusive Human Intention Prediction with Wearable Sensors: Machine Learning Techniques for the Reaching Task Use Case , 2020 .

[3]  Zhe Wang,et al.  sEMG Based Human Motion Intention Recognition , 2019, J. Robotics.

[4]  Bo Li,et al.  Upper body motion recognition based on key frame and random forest regression , 2018, Multimedia Tools and Applications.

[5]  Piotr Gawron,et al.  Natural hand gestures for human identification in a Human-Computer Interface , 2014, 2014 4th International Conference on Image Processing Theory, Tools and Applications (IPTA).

[6]  Monica Tiboni,et al.  Preliminary Validation of a Device for the Upper and Lower Limb Robotic Rehabilitation , 2019, 2019 23rd International Conference on Mechatronics Technology (ICMT).

[7]  A. Roby-Brami,et al.  Influence of the side of brain damage on postural upper-limb control including the scapula in stroke patients , 2012, Experimental Brain Research.

[8]  Cristina Nuzzi,et al.  Deep Learning Based Machine Vision: First Steps Towards a Hand Gesture Recognition Set Up for Collaborative Robots , 2018, 2018 Workshop on Metrology for Industry 4.0 and IoT.

[9]  M. Nixon,et al.  Automated Human Recognition by Gait using Neural Network , 2008, 2008 First Workshops on Image Processing Theory, Tools and Applications.

[10]  Qing Li,et al.  Robot recognizing humans intention and interacting with humans based on a multi-task model combining ST-GCN-LSTM model and YOLO model , 2021, Neurocomputing.

[11]  Nooritawati Md Tahir,et al.  Classification of autism children gait patterns using Neural Network and Support Vector Machine , 2016, 2016 IEEE Symposium on Computer Applications & Industrial Electronics (ISCAIE).

[12]  A K Bourke,et al.  Activity classification using a single chest mounted tri-axial accelerometer. , 2011, Medical engineering & physics.

[13]  Yimin Zhou,et al.  A novel finger and hand pose estimation technique for real-time hand gesture recognition , 2016, Pattern Recognit..

[14]  Angelo Cangelosi,et al.  Human?Robot Interaction and Neuroprosthetics: A review of new technologies. , 2017, IEEE Consumer Electronics Magazine.

[15]  Monica Tiboni,et al.  Multi-Sensor Validation Approach of an End-Effector-Based Robot for the Rehabilitation of the Upper and Lower Limb , 2020, Electronics.

[16]  Mauro Serpelloni,et al.  ERRSE: Elbow Robotic Rehabilitation System with an EMG-Based Force Control , 2017, RAAD.

[17]  S. Negrini,et al.  Quantitative analysis of voluntary movement and anticipatory postural adjustments: a functional approach , 2021, Computer methods in biomechanics and biomedical engineering.

[18]  Giovanni Cannaviello,et al.  Exoskeleton and End‐Effector Robots for Upper and Lower Limbs Rehabilitation: Narrative Review , 2018, PM & R : the journal of injury, function, and rehabilitation.

[19]  Wendy Moyle,et al.  Potential of telepresence robots to enhance social connectedness in older adults with dementia: an integrative review of feasibility , 2017, International Psychogeriatrics.

[20]  Sandeep Pirbhulal,et al.  A comparative study of motion recognition methods for efficacy assessment of upper limb function , 2018, International Journal of Adaptive Control and Signal Processing.

[21]  Xingang Zhao,et al.  Deep Learning for EMG-based Human-Machine Interaction: A Review , 2021, IEEE/CAA Journal of Automatica Sinica.

[22]  Cheng Yang,et al.  Upper limb movement analysis via marker tracking with a single-camera system , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[23]  Alberto Borboni,et al.  Effects of Soft Tissue Artifact in the Measurement of Hand Kinematics , 2020 .

[24]  Guillaume Morel,et al.  Real-time human posture observation from a small number of joint measurements , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[25]  Mauro Serpelloni,et al.  Use of Wearable Inertial Sensor in the Assessment of Timed-Up-and-Go Test: Influence of Device Placement on Temporal Variable Estimation , 2016, MobiHealth.

[26]  Agnès Roby-Brami,et al.  Can We Achieve Intuitive Prosthetic Elbow Control Based on Healthy Upper Limb Motor Strategies? , 2018, Front. Neurorobot..

[27]  Tao Liu,et al.  Gait Analysis Using Wearable Sensors , 2012, Sensors.

[28]  B. Garcia-Zapirain,et al.  Deep Learning Techniques Applied to Predict and Measure Finger Movement in Patients with Multiple Sclerosis , 2021, Applied Sciences.

[29]  Etienne Burdet,et al.  Is EMG a Viable Alternative to BCI for Detecting Movement Intention in Severe Stroke? , 2018, IEEE Transactions on Biomedical Engineering.

[30]  Lin Zhang,et al.  Deep-Learning-Based Human Intention Prediction Using RGB Images and Optical Flow , 2020, J. Intell. Robotic Syst..

[31]  D. Kumar,et al.  Deep learning in Human Gait Recognition: An Overview , 2021, 2021 International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE).

[32]  Ezio Preatoni,et al.  Supervised Machine Learning Applied to Wearable Sensor Data Can Accurately Classify Functional Fitness Exercises Within a Continuous Workout , 2020, Frontiers in Bioengineering and Biotechnology.

[33]  Wolfram Burgard,et al.  Graph-Based Action Models for Human Motion Classification , 2012, ROBOTIK.

[34]  B. Bussel,et al.  Motor compensation and recovery for reaching in stroke patients , 2003, Acta neurologica Scandinavica.