Auditory feedback in tele-rehabilitation based on automated gait classification

We describe a proof-of-concept for the implementation of a mobile auditory biofeedback system based on automated classification of functional gait disorders. The classification is embedded in a sensor-instrumented insole and is based on ground reaction forces (GRFs). GRF data have been successfully used for the classification of gait patterns into clinically relevant classes and are frequently used in clinical practice to quantitatively describe human motion. A feed-forward neural network that was implemented on the firmware of the insole is used to estimate the GRFs using pressure and accelerator data. Compared to GRF measurements obtained from force plates, the estimated GRFs performed highly accurately. To distinguish between normal physiological gait and gait disorders, we trained and evaluated a support vector machine with labeled data from a publicly accessible database. The automated gait classification was sonified for auditory feedback. The high potential of the implemented auditory feedback for preventive and supportive applications in physical therapy, such as supervised therapy settings and tele-rehabilitation, was highlighted by a semi-structured interview with two experts.

[1]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[2]  J. Nadal,et al.  Application of principal component analysis in vertical ground reaction force to discriminate normal and abnormal gait. , 2009, Gait & posture.

[3]  Brian Horsak,et al.  GaitRec, a large-scale ground reaction force dataset of healthy and impaired gait , 2020, Scientific Data.

[4]  Panos Markopoulos,et al.  Design and Evaluation of SONIS, a Wearable Biofeedback System for Gait Retraining , 2020 .

[5]  Yuan Peng,et al.  Walking Gait Phase Detection Based on Acceleration Signals Using LSTM-DNN Algorithm , 2019 .

[6]  Stacy J. Morris Bamberg,et al.  Instrumented insole vs. force plate: A comparison of center of plantar pressure , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[7]  Arnold Baca,et al.  Automatic Classification of Functional Gait Disorders , 2017, IEEE Journal of Biomedical and Health Informatics.

[8]  Matthew W. M. Rodger,et al.  Synthesis of Walking Sounds for Alleviating Gait Disturbances in Parkinson's Disease , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[9]  Alexander Meschtscherjakov,et al.  Sonification approaches in sports in the past decade: a literature review , 2020, Audio Mostly Conference.

[10]  Jeffrey E. Boyd,et al.  CORRECTIVE SONIC FEEDBACK FOR SPEED SKATING: A CASE STUDY , 2010 .

[11]  Y. Baram,et al.  Auditory feedback control for improvement of gait in patients with Multiple Sclerosis , 2007, Journal of the Neurological Sciences.

[12]  K Aminian,et al.  Ambulatory assessment of 3D ground reaction force using plantar pressure distribution. , 2010, Gait & posture.

[13]  Anita Kiselka,et al.  Short-Term Effects of Real-Time Auditory Display (Sonification) on Gait Parameters in People with Parkinsons’ Disease—A Pilot Study , 2017 .

[14]  Christopher J. C. Burges,et al.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998, Data Mining and Knowledge Discovery.

[15]  Carole A. Tucker,et al.  Measuring Walking: A Handbook of Clinical Gait Analysis , 2014 .

[16]  R. Magill.  Motor learning and control : concepts and applications , 2004 .

[17]  Thompson Sarkodie-Gyan,et al.  Automatic classification of pathological gait patterns using ground reaction forces and machine learning algorithms , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[18]  Elham Monaghesh,et al.  The role of telehealth during COVID-19 outbreak: a systematic review based on current evidence , 2020, BMC Public Health.

[19]  Andy Farnell,et al.  Designing Sound , 2008 .

[20]  Toshiki Kobayashi,et al.  Kinetic Gait Analysis Using a Low-Cost Insole , 2013, IEEE Transactions on Biomedical Engineering.

[21]  Jesper Hohagen,et al.  Bewegungssonifikation: Psychologische Grundlagen und Auswirkungen der Verklanglichung menschlicher Handlungen in der Rehabilitation, im Sport und bei Musikaufführungen , 2019, Jahrbuch Musikpsychologie.

[22]  Luca Turchet,et al.  Custom made wireless systems for interactive footstep sounds synthesis , 2014 .

[23]  Nicola Vitiello,et al.  A Wireless Pressure-sensitive Insole for Gait Analysis , 2012 .

[24]  R. Riener,et al.  Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review , 2012, Psychonomic Bulletin & Review.

[25]  G. Wulf,et al.  Choose to move: The motivational impact of autonomy support on motor learning , 2015, Psychonomic bulletin & review.

[26]  K. An,et al.  Estimate of the Optimum Cutoff Frequency for the Butterworth Low-Pass Digital Filter , 1999 .

[27]  Stephen Hayes,et al.  The Classification of Minor Gait Alterations Using Wearable Sensors and Deep Learning , 2019, IEEE Transactions on Biomedical Engineering.

[28]  Xingda Qu,et al.  Estimation of Foot Plantar Center of Pressure Trajectories with Low-Cost Instrumented Insoles Using an Individual-Specific Nonlinear Model , 2018, Sensors.

[29]  D. Gouwanda,et al.  ANN for gait estimations: A review on current trends and future applications , 2016, 2016 IEEE EMBS Conference on Biomedical Engineering and Sciences (IECBES).

[30]  Luca Turchet,et al.  Footstep sounds synthesis: Design, implementation, and evaluation of foot–floor interactions, surface materials, shoe types, and walkers’ features , 2016 .

[31]  Seung Eel Oh,et al.  Predicting Complete Ground Reaction Forces and Moments During Gait With Insole Plantar Pressure Information Using a Wavelet Neural Network. , 2015, Journal of biomechanical engineering.

[32]  Seong-Hi Park,et al.  Tools for assessing fall risk in the elderly: a systematic review and meta-analysis , 2017, Aging Clinical and Experimental Research.

[33]  Iván González,et al.  An Ambulatory System for Gait Monitoring Based on Wireless Sensorized Insoles , 2015, Sensors.

[34]  R J Abboud,et al.  The Pedar in-shoe system: repeatability and normal pressure values. , 2007, Gait & posture.

[35]  Nina Schaffert,et al.  Acoustic feedback training in adaptive rowing , 2012 .

[36]  Ali Momeni,et al.  Ml.lib: robust, cross-platform, open-source machine learning for max and pure data , 2015, NIME.

[37]  Jung Kim,et al.  Flexible insole ground reaction force measurement shoes for jumping and running , 2016, 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[38]  Lee Smith,et al.  The use of sonification for physiotherapy in human movement tasks: A scoping review , 2020 .

[39]  M. Barry,et al.  Shared decision making--pinnacle of patient-centered care. , 2012, The New England journal of medicine.

[40]  Alberto J. Palma,et al.  Embedded sensor insole for wireless measurement of gait parameters , 2013, Australasian Physical & Engineering Sciences in Medicine.

[41]  M. Wimmer,et al.  Unsupervised gait retraining using a wireless pressure-detecting shoe insole. , 2019, Gait & posture.

[42]  K. Götz-Neumann.  Gehen verstehen: Ganganalyse in der Physiotherapie , 2016 .

[43]  Thomas Jöllenbeck,et al.  Gait Training in Orthopedic Rehabilitation after Joint Replacement - Back to Normal Gait with Sonification? , 2019, Int. J. Comput. Sci. Sport.

[44]  Parker G Rosquist.  Modeling Three Dimensional Ground Reaction Force Using Nanocomposite Piezoresponsive Foam Sensors , 2017 .

[45]  Carlo Fischione,et al.  The Internet of Audio Things: State of the Art, Vision, and Challenges , 2020, IEEE Internet of Things Journal.

[46]  Luca Turchet,et al.  Interactive sonification and the IoT: the case of smart sonic shoes for clinical applications , 2019, Audio Mostly Conference.

[47]  Anita Kiselka,et al.  SONIGait: a wireless instrumented insole device for real-time sonification of gait , 2015, Journal on Multimodal User Interfaces.

[48]  F C T van der Helm,et al.  Use of pressure insoles to calculate the complete ground reaction forces. , 2004, Journal of biomechanics.

[49]  Joseph A. Paradiso,et al.  The gesture recognition toolkit , 2014, J. Mach. Learn. Res..

[50]  Rainer Wieching,et al.  Disentangling the health benefits of walking from increased exposure to falls in older people using remote gait monitoring and multi-dimensional analysis , 2017, Physiological measurement.