Recognition of Walking Activity and Prediction of Gait Periods with a CNN and First-Order MC Strategy

In this paper, a strategy for recognition of human walking activities and prediction of gait periods using wearable sensors is presented. First, a Convolutional Neural Network (CNN) is developed for the recognition of three walking activities (level-ground walking, ramp ascent and descent) and recognition of gait periods. Second, a first-order Markov Chain (MC) is employed for the prediction of gait periods, based on the observation of decisions made by the CNN for each walking activity. The validation of the proposed methods is performed using data from three inertial measurement units (IMU) attached to the lower limbs of participants. The results show that the CNN, together with the first-order MC, achieves mean accuracies of 100% and 98.32% for recognition of walking activities and gait periods, respectively. Prediction of gait periods are achieved with mean accuracies of 99.78%, 97.56% and 97.35% during level-ground walking, ramp ascent and descent, respectively. Overall, the benefits of our work for accurate recognition and prediction of walking activity and gait periods, make it a suitable high-level method for the development of intelligent assistive robots.

[1]  Gerald Penn,et al.  Convolutional Neural Networks for Speech Recognition , 2014, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[2]  Uriel Martinez-Hernandez,et al.  Adaptive Bayesian inference system for recognition of walking activities and prediction of gait events using wearable sensors , 2018, Neural Networks.

[3]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[4]  Shyamal Patel,et al.  A review of wearable sensors and systems with application in rehabilitation , 2012, Journal of NeuroEngineering and Rehabilitation.

[5]  Hyun Chul Cho,et al.  Online Estimation of Dynamic Bayesian Network Parameter , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[6]  Dilip Sarkar,et al.  Log-Sum Distance Measures and Its Application to Human-Activity Monitoring and Recognition Using Data From Motion Sensors , 2017, IEEE Sensors Journal.

[7]  Rodney A. Brooks,et al.  A Robust Layered Control Syste For A Mobile Robot , 2022 .

[8]  Aaron J. Young,et al.  Analysis of using EMG and mechanical sensors to enhance intent recognition in powered lower limb prostheses , 2014, Journal of neural engineering.

[9]  Christopher Kirtley,et al.  Clinical Gait Analysis: Theory and Practice , 2006 .

[10]  Kuan Zhang,et al.  Assessment of human locomotion by using an insole measurement system and artificial neural networks. , 2005, Journal of biomechanics.

[11]  Uriel Martinez-Hernandez,et al.  Probabilistic identification of sit-to-stand and stand-to-sit with a wearable sensor , 2019, Pattern Recognit. Lett..

[12]  Svetha Venkatesh,et al.  Activity recognition and abnormality detection with the switching hidden semi-Markov model , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[13]  Huosheng Hu,et al.  Human motion tracking for rehabilitation - A survey , 2008, Biomed. Signal Process. Control..

[14]  Michael Goldfarb,et al.  Multiclass Real-Time Intent Recognition of a Powered Lower Limb Prosthesis , 2010, IEEE Transactions on Biomedical Engineering.

[15]  Nicholas P. Fey,et al.  Intent Recognition in a Powered Lower Limb Prosthesis Using Time History Information , 2013, Annals of Biomedical Engineering.

[16]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[17]  Uriel Martinez-Hernandez,et al.  Simultaneous Bayesian Recognition of Locomotion and Gait Phases With Wearable Sensors , 2018, IEEE Sensors Journal.

[18]  George Panoutsos,et al.  Interval Type-2 Radial Basis Function Neural Network: A Modeling Framework , 2015, IEEE Transactions on Fuzzy Systems.

[19]  Levi J. Hargrove,et al.  A Classification Method for User-Independent Intent Recognition for Transfemoral Amputees Using Powered Lower Limb Prostheses , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[20]  George Panoutsos,et al.  Evolutionary Extreme Learning Machine for the Interval Type-2 Radial Basis Function Neural Network: A Fuzzy Modelling Approach , 2018, 2018 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).

[21]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[22]  Uriel Martinez-Hernandez,et al.  Multisensory Wearable Interface for Immersion and Telepresence in Robotics , 2017, IEEE Sensors Journal.

[23]  Tony J. Dodd,et al.  Active sensorimotor control for tactile exploration , 2017, Robotics Auton. Syst..

[24]  Robert J. Wood,et al.  Wearable soft sensing suit for human gait measurement , 2014, Int. J. Robotics Res..

[25]  Krishna R. Pattipati,et al.  A hidden Markov model-based algorithm for fault diagnosis with partial and imperfect tests , 2000, IEEE Trans. Syst. Man Cybern. Part C.

[26]  Fan Zhang,et al.  Continuous Locomotion-Mode Identification for Prosthetic Legs Based on Neuromuscular–Mechanical Fusion , 2011, IEEE Transactions on Biomedical Engineering.

[27]  D. Weber,et al.  The role of assistive robotics in the lives of persons with disability. , 2010, American journal of physical medicine & rehabilitation.

[28]  Tony J. Dodd,et al.  Feeling the Shape: Active Exploration Behaviors for Object Recognition With a Robotic Hand , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[29]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.