Unsupervised Cross-Subject Adaptation for Predicting Human Locomotion Intent

Accurately predicting human locomotion intent is beneficial in controlling wearable robots and in assisting humans to walk smoothly on different terrains. Traditional methods for predicting human locomotion intent require collecting and labeling the human signals, and training specific classifiers for each new subject, which introduce a heavy burden on both the subject and the researcher. In addressing this issue, the present study liberates the subject and the researcher from labeling a large amount of data, by incorporating an unsupervised cross-subject adaptation method to predict the locomotion intent of a target subject whose signals are not labeled. The adaptation is realized by designing two classifiers to maximize the classification discrepancy and a feature generator to align the hidden features of the source and the target subjects to minimize the classification discrepancy. A neural network is trained by the labeled training set of source subjects and the unlabeled training set of target subjects. Then it is validated and tested on the validation set and the test set of target subjects. Experimental results in the leave-one-subject-out test indicate that the present method can classify the locomotion intent and activities of target subjects at the averaged accuracy of 93.60% and 94.59% on two public datasets. The present method increases the user-independence of the classifiers, but it has been evaluated only on the data of subjects without disabilities. The potential of the present method to predict the locomotion intent of subjects with disabilities and control the wearable robots will be evaluated in future work.

[1]  Daniel P. Ferris,et al.  State of the Art and Future Directions for Lower Limb Robotic Exoskeletons , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[2]  Qining Wang,et al.  Real-Time On-Board Recognition of Continuous Locomotion Modes for Amputees With Robotic Transtibial Prostheses , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[3]  Ken Chen,et al.  Yoga Posture Recognition and Quantitative Evaluation with Wearable Sensors Based on Two-Stage Classifier and Prior Bayesian Network , 2019, Sensors.

[4]  J. Decety,et al.  From the perception of action to the understanding of intention , 2001, Nature reviews. Neuroscience.

[5]  George Trigeorgis,et al.  Domain Separation Networks , 2016, NIPS.

[6]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[7]  Gary Burnett,et al.  Translational Motion Tracking of Leg Joints for Enhanced Prediction of Walking Tasks , 2018, IEEE Transactions on Biomedical Engineering.

[8]  Omid Dehzangi,et al.  IMU-Based Gait Recognition Using Convolutional Neural Networks and Multi-Sensor Fusion , 2017, Sensors.

[9]  Yiqiang Chen,et al.  Deep Transfer Learning for Cross-domain Activity Recognition , 2018, ICCSE'18.

[10]  Ken Chen,et al.  Design and Basic Control of Extra Robotic Legs for Dynamic Walking Assistance , 2019, 2019 IEEE International Conference on Advanced Robotics and its Social Impacts (ARSO).

[11]  Levi J. Hargrove,et al.  Benchmark Datasets for Bilateral Lower-Limb Neuromechanical Signals from Wearable Sensors during Unassisted Locomotion in Able-Bodied Individuals , 2018, Front. Robot. AI.

[12]  Tatsuya Harada,et al.  Maximum Classifier Discrepancy for Unsupervised Domain Adaptation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[13]  Wen Zhang,et al.  Environmental Features Recognition for Lower Limb Prostheses Toward Predictive Walking , 2019, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[14]  Nicola Vitiello,et al.  CYBERLEGs: A User-Oriented Robotic Transfemoral Prosthesis with Whole-Body Awareness Control , 2014, IEEE Robotics & Automation Magazine.

[15]  Levi J. Hargrove,et al.  Fusion of Bilateral Lower-Limb Neuromechanical Signals Improves Prediction of Locomotor Activities , 2018, Front. Robot. AI.

[16]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[17]  Kuangen Zhang,et al.  Sequential Decision Fusion for Environmental Classification in Assistive Walking , 2019, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[18]  Mazharul Islam,et al.  Detection of Gait Modes Using an Artificial Neural Network during Walking with a Powered Ankle-Foot Orthosis , 2016, Journal of biophysics.

[19]  Clarence W. de Silva,et al.  Sensor Fusion for Predictive Control of Human-Prosthesis-Environment Dynamics in Assistive Walking: A Survey , 2019, ArXiv.

[20]  Jing Wang,et al.  Unsupervised Domain Adaptation Learning Algorithm for RGB-D Staircase Recognition , 2019, ArXiv.

[21]  Robert Riener,et al.  Control strategies for active lower extremity prosthetics and orthotics: a review , 2015, Journal of NeuroEngineering and Rehabilitation.

[22]  Billur Barshan,et al.  Recognizing Daily and Sports Activities in Two Open Source Machine Learning Environments Using Body-Worn Sensor Units , 2014, Comput. J..

[23]  Michael Goldfarb,et al.  A Phase Variable Approach for IMU-Based Locomotion Activity Recognition , 2018, IEEE Transactions on Biomedical Engineering.

[24]  Huseyin Atakan Varol,et al.  User-Independent Intent Recognition for Lower Limb Prostheses Using Depth Sensing , 2018, IEEE Transactions on Biomedical Engineering.

[25]  Shriya S Srinivasan,et al.  Proprioception from a neurally controlled lower-extremity prosthesis , 2018, Science Translational Medicine.

[26]  Trevor Darrell,et al.  Adversarial Discriminative Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[28]  Michael Goldfarb,et al.  Design and Control of a Powered Transfemoral Prosthesis , 2008, Int. J. Robotics Res..

[29]  Ken Chen,et al.  Self-Supervised Learning for Specified Latent Representation , 2020, IEEE Transactions on Fuzzy Systems.

[30]  Levi J. Hargrove,et al.  Using bilateral lower limb kinematic and myoelectric signals to predict locomotor activities: A pilot study , 2017, 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER).

[31]  Ken Chen,et al.  Smoother-Based 3-D Foot Trajectory Estimation Using Inertial Sensors , 2019, IEEE Transactions on Biomedical Engineering.

[32]  Fan Zhang,et al.  Continuous Locomotion-Mode Identification for Prosthetic Legs Based on Neuromuscular–Mechanical Fusion , 2011, IEEE Transactions on Biomedical Engineering.

[33]  He Huang,et al.  A Strategy for Identifying Locomotion Modes Using Surface Electromyography , 2009, IEEE Transactions on Biomedical Engineering.

[34]  Enhao Zheng,et al.  Noncontact Capacitive Sensing-Based Locomotion Transition Recognition for Amputees With Robotic Transtibial Prostheses , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[35]  Koby Crammer,et al.  A theory of learning from different domains , 2010, Machine Learning.

[36]  Hassan Ghasemzadeh,et al.  Personalization without User Interruption: Boosting Activity Recognition in New Subjects Using Unlabeled Data , 2017, 2017 ACM/IEEE 8th International Conference on Cyber-Physical Systems (ICCPS).