Human Activity Recognition of Individuals with Lower Limb Amputation in Free-Living Conditions: A Pilot Study

This pilot study aimed to investigate the implementation of supervised classifiers and a neural network for the recognition of activities carried out by Individuals with Lower Limb Amputation (ILLAs), as well as individuals without gait impairment, in free living conditions. Eight individuals with no gait impairments and four ILLAs wore a thigh-based accelerometer and walked on an improvised route in the vicinity of their homes across a variety of terrains. Various machine learning classifiers were trained and tested for recognition of walking activities. Additional investigations were made regarding the detail of the activity label versus classifier accuracy and whether the classifiers were capable of being trained exclusively on non-impaired individuals’ data and could recognize physical activities carried out by ILLAs. At a basic level of label detail, Support Vector Machines (SVM) and Long-Short Term Memory (LSTM) networks were able to acquire 77–78% mean classification accuracy, which fell with increased label detail. Classifiers trained on individuals without gait impairment could not recognize activities carried out by ILLAs. This investigation presents the groundwork for a HAR system capable of recognizing a variety of walking activities, both for individuals with no gait impairments and ILLAs.

[1]  P. Dijkstra,et al.  Barriers and Facilitators of Participation in Sports: A Qualitative Study on Dutch Individuals with Lower Limb Amputation , 2013, PloS one.

[2]  Mehdi Hussain,et al.  What Lies Beneath One’s Feet? Terrain Classification Using Inertial Data of Human Walk , 2019, Applied Sciences.

[3]  Wen Zhang,et al.  Environmental Features Recognition for Lower Limb Prostheses Toward Predictive Walking , 2019, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[4]  Qining Wang,et al.  Implementing a SoC-FPGA Based Acceleration System for On-Board SVM Training for Robotic Transtibial Prostheses , 2018, 2018 IEEE International Conference on Real-time Computing and Robotics (RCAR).

[5]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[6]  Kamran Raza,et al.  Effect of Feature Selection, SMOTE and under Sampling on Class Imbalance Classification , 2012, 2012 UKSim 14th International Conference on Computer Modelling and Simulation.

[7]  Alexander Horsch,et al.  Separating Movement and Gravity Components in an Acceleration Signal and Implications for the Assessment of Human Daily Physical Activity , 2013, PloS one.

[8]  Eirik Vågeskar.  Activity Recognition for Stroke Patients , 2017 .

[9]  David Howard,et al.  A Comparison of Feature Extraction Methods for the Classification of Dynamic Activities From Accelerometer Data , 2009, IEEE Transactions on Biomedical Engineering.

[10]  Abdulhamit Subasi,et al.  Sensor Based Human Activity Recognition Using Adaboost Ensemble Classifier , 2018 .

[11]  C. Granger,et al.  Physical activity participation amongst individuals with lower limb amputation , 2019, Disability and rehabilitation.

[12]  Tim Dallas,et al.  Feature Selection and Activity Recognition System Using a Single Triaxial Accelerometer , 2014, IEEE Transactions on Biomedical Engineering.

[13]  Giovanni Pau,et al.  Acceleration Feature Extraction of Human Body Based on Wearable Devices , 2021, Energies.

[14]  J. Bussmann,et al.  Daily physical activity and heart rate response in people with a unilateral transtibial amputation for vascular disease. , 2004, Archives of physical medicine and rehabilitation.

[15]  Andrew McDaid,et al.  Moving the Lab into the Mountains: A Pilot Study of Human Activity Recognition in Unstructured Environments , 2021, Sensors.

[16]  Gary Burnett,et al.  Translational Motion Tracking of Leg Joints for Enhanced Prediction of Walking Tasks , 2018, IEEE Transactions on Biomedical Engineering.

[17]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[18]  O. Sarbishei,et al.  A Quantitative Comparison of Overlapping and Non-Overlapping Sliding Windows for Human Activity Recognition Using Inertial Sensors , 2019, Sensors.

[19]  Kuangen Zhang,et al.  Sequential Decision Fusion for Environmental Classification in Assistive Walking , 2019, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[20]  Gabriella Balestra,et al.  Comparison of Different Sets of Features for Human Activity Recognition by Wearable Sensors , 2018, Sensors.

[21]  M. N. Nyan,et al.  Classification of gait patterns in the time-frequency domain. , 2006, Journal of biomechanics.

[22]  Lu Xu,et al.  Human activity recognition based on random forests , 2017, 2017 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD).

[23]  Daphne Wezenberg,et al.  Relation between aerobic capacity and walking ability in older adults with a lower-limb amputation. , 2013, Archives of physical medicine and rehabilitation.

[24]  T Togawa,et al.  Classification of waist-acceleration signals in a continuous walking record. , 2000, Medical engineering & physics.

[25]  G. Kitas,et al.  Measurement of sedentary time and physical activity in rheumatoid arthritis: an ActiGraph and activPAL™ validation study , 2020, Rheumatology International.

[26]  Min Sheng,et al.  A CNN-Based Method for Intent Recognition Using Inertial Measurement Units and Intelligent Lower Limb Prosthesis , 2019, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[27]  Takeshi Nishida,et al.  Deep recurrent neural network for mobile human activity recognition with high throughput , 2017, Artificial Life and Robotics.

[28]  Measuring the Daily Stepping Activity of People with Transtibial Amputation Using the ActivPAL™ Activity Monitor , 2014 .

[29]  Yu Zhao,et al.  Deep Residual Bidir-LSTM for Human Activity Recognition Using Wearable Sensors , 2017, Mathematical Problems in Engineering.

[30]  Davide Anguita,et al.  Human Activity Recognition on Smartphones Using a Multiclass Hardware-Friendly Support Vector Machine , 2012, IWAAL.

[31]  Paul Lukowicz,et al.  Collecting complex activity datasets in highly rich networked sensor environments , 2010, 2010 Seventh International Conference on Networked Sensing Systems (INSS).

[32]  Aleksey Khlyupin,et al.  Human Activity Recognition using Continuous Wavelet Transform and Convolutional Neural Networks , 2021, ArXiv.

[33]  Bruce G. Marcot,et al.  What is an optimal value of k in k-fold cross-validation in discrete Bayesian network analysis? , 2020, Computational Statistics.

[34]  Laura Frølich,et al.  Removal of muscular artifacts in EEG signals: a comparison of linear decomposition methods , 2018, Brain Informatics.

[35]  Mi Zhang,et al.  A feature selection-based framework for human activity recognition using wearable multimodal sensors , 2011, BODYNETS.

[36]  Shu Wang,et al.  Learning structures of interval-based Bayesian networks in probabilistic generative model for human complex activity recognition , 2018, Pattern Recognit..

[37]  Rienk Dekker,et al.  Amputees and Sports , 2011, Sports medicine.

[38]  Robert X. Gao,et al.  SVM-based multi-sensor fusion for free-living physical activity assessment , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[39]  Hakob Sarukhanyan,et al.  ACTIVITY RECOGNITION USING K-NEAREST NEIGHBOR ALGORITHM ON SMARTPHONE WITH TRI-AXIAL ACCELEROMETER , 2012 .

[40]  Fernando Fernández Martínez,et al.  Feature extraction from smartphone inertial signals for human activity segmentation , 2016, Signal Process..

[41]  John Staudenmayer,et al.  The activPALTM Accurately Classifies Activity Intensity Categories in Healthy Adults , 2017, Medicine and science in sports and exercise.

[42]  M. Akay,et al.  Discrimination of walking patterns using wavelet-based fractal analysis , 2002, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[43]  Laurence Kenney,et al.  Technology for monitoring everyday prosthesis use: a systematic review , 2020, Journal of NeuroEngineering and Rehabilitation.

[44]  C. Hanson,et al.  Effect of Participation in Physical Activity on Body Image of Amputees , 2002, American journal of physical medicine & rehabilitation.

[45]  Jaesoon Choi,et al.  Measurement and Analysis of Gait Pattern during Stair Walk for Improvement of Robotic Locomotion Rehabilitation System , 2019, Applied bionics and biomechanics.

[46]  Richard R Neptune,et al.  Identifying classifier input signals to predict a cross-slope during transtibial amputee walking , 2018, PloS one.

[47]  Gongping Yang,et al.  On the Class Imbalance Problem , 2008, 2008 Fourth International Conference on Natural Computation.

[48]  B. Heller,et al.  Recognizing Human Activity in Free-Living Using Multiple Body-Worn Accelerometers , 2017, IEEE Sensors Journal.

[49]  Rafiqul Zaman Khan,et al.  Methods to Avoid Over-Fitting and Under-Fitting in Supervised Machine Learning (Comparative Study) , 2014 .

[50]  M. Plotnik,et al.  The effect of uphill and downhill walking on gait parameters: A self-paced treadmill study. , 2017, Journal of biomechanics.

[51]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[52]  Qining Wang,et al.  Performance analysis of hardware acceleration for locomotion mode recognition in robotic prosthetic control , 2018, 2018 IEEE International Conference on Cyborg and Bionic Systems (CBS).

[53]  Josef Hallberg,et al.  Automatic Annotation for Human Activity Recognition in Free Living Using a Smartphone , 2018, Sensors.

[54]  Alberto G. Bonomi,et al.  Identifying Types of Physical Activity With a Single Accelerometer: Evaluating Laboratory-trained Algorithms in Daily Life , 2011, IEEE Transactions on Biomedical Engineering.

[55]  M. Granat,et al.  The validation of a novel activity monitor in the measurement of posture and motion during everyday activities , 2006, British Journal of Sports Medicine.

[56]  Im Y. Jung,et al.  A review of privacy-preserving human and human activity recognition , 2020, International Journal on Smart Sensing and Intelligent Systems.

[57]  A. F. Salleh,et al.  GROUND REACTION FORCE RESPONSE DURING RUNNING ON DIFFERENT SURFACE HARDNESS , 2017 .

[58]  T Togawa,et al.  Classification of acceleration waveforms during walking by wavelet transform. , 1997, Methods of information in medicine.

[59]  Gert R. G. Lanckriet,et al.  Multi-sensor physical activity recognition in free-living , 2014, UbiComp Adjunct.

[60]  J. Haselkorn,et al.  Physical activity barriers and enablers in older Veterans with lower-limb amputation. , 2014, Journal of rehabilitation research and development.

[61]  Tom Fearn.  Double Cross-Validation , 2010 .

[62]  Chris D. Nugent,et al.  A Public Domain Dataset for Human Activity Recognition in Free-Living Conditions , 2019, 2019 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI).

[63]  Floriant Labarrière,et al.  Machine Learning Approaches for Activity Recognition and/or Activity Prediction in Locomotion Assistive Devices—A Systematic Review , 2020, Sensors.

[64]  Andrew Christian Flores,et al.  An Evaluation of SVM and Naive Bayes with SMOTE on Sentiment Analysis Data Set , 2018, 2018 International Conference on Engineering, Applied Sciences, and Technology (ICEAST).

[65]  B. Hu,et al.  Machine learning algorithms based on signals from a single wearable inertial sensor can detect surface- and age-related differences in walking. , 2018, Journal of biomechanics.

[66]  Yoshua Bengio,et al.  Practical Recommendations for Gradient-Based Training of Deep Architectures , 2012, Neural Networks: Tricks of the Trade.

[67]  Anthony McGarry,et al.  Motivations and barriers to prosthesis users participation in physical activity, exercise and sport: a review of the literature , 2012, Prosthetics and orthotics international.

[68]  B. Celler,et al.  Accelerometry Based Classification of Walking Patterns Using Time-frequency Analysis , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[69]  Lin Tan,et al.  Sequential Human Activity Recognition Based on Deep Convolutional Network and Extreme Learning Machine Using Wearable Sensors , 2018, J. Sensors.