Freezing of gait assessment with inertial measurement units and deep learning: effect of tasks, medication states, and stops

Background: Freezing of gait (FOG) is an episodic and highly disabling symptom of Parkinson's Disease (PD). Traditionally, FOG assessment relies on time-consuming visual inspection of camera footage. Therefore, previous studies have proposed portable and automated solutions to annotate FOG. However, automated FOG assessment is challenging due to gait variability caused by medication effects and varying FOG-provoking tasks. Moreover, whether automated approaches can differentiate FOG from typical everyday movements, such as volitional stops, remains to be determined. To address these questions, we evaluated an automated FOG assessment model with deep learning (DL) based on inertial measurement units (IMUs). We assessed its performance trained on all standardized FOG-provoking tasks and medication states, as well as on specific tasks and medication states. Furthermore, we examined the effect of adding stopping periods on FOG detection performance. Methods: Twelve PD patients with self-reported FOG (mean age 69.33 +/- 6.28 years) completed a FOG-provoking protocol, including timed-up-and-go and 360-degree turning-in-place tasks in On/Off dopaminergic medication states with/without volitional stopping. IMUs were attached to the pelvis and both sides of the tibia and talus. A multi-stage temporal convolutional network was developed to detect FOG episodes. FOG severity was quantified by the percentage of time frozen (%TF) and the number of freezing episodes (#FOG). The agreement between the model-generated outcomes and the gold standard experts' video annotation was assessed by the intra-class correlation coefficient (ICC). Results: For FOG assessment in trials without stopping, the agreement of our model was strong (ICC(%TF) = 0.92 [0.68, 0.98]; ICC(#FOG) = 0.95 [0.72, 0.99]). Models trained on a specific FOG-provoking task could not generalize to unseen tasks, while models trained on a specific medication state could generalize to unseen states. For assessment in trials with stopping, the model trained on stopping trials made fewer false positives than the model trained without stopping (ICC(%TF) = 0.95 [0.73, 0.99]; ICC(#FOG) = 0.79 [0.46, 0.94]). Conclusion: A DL model trained on IMU signals allows valid FOG assessment in trials with/without stops containing different medication states and FOG-provoking tasks. These results are encouraging and enable future work investigating automated FOG assessment during everyday life.

[1]  H. Nguyen,et al.  Predicting the Onset of Freezing of Gait Using EEG Dynamics , 2022, Applied Sciences.

[2]  M. Plotnik,et al.  Bilateral Leg Stepping Coherence as a Predictor of Freezing of Gait in Patients With Parkinson’s Disease Walking With Wearable Sensors , 2022, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[3]  B. Vanrumste,et al.  Drinking Gesture Detection Using Wrist-Worn IMU Sensors with Multi-Stage Temporal Convolutional Network in Free-Living Environments , 2022, 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC).

[4]  W. Chen,et al.  Can Wearable Devices and Machine Learning Techniques Be Used for Recognizing and Segmenting Modified Physical Performance Test Items? , 2022, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[5]  H. Kadone,et al.  Measurement Accuracy of Freezing of Gait Scoring Based on Videos , 2022, Frontiers in Human Neuroscience.

[6]  G. Stebbins,et al.  Validation of the Thai Version of the Movement Disorder Society-Sponsored Revision of the Unified Parkinson's Disease Rating Scale , 2022, Journal of movement disorders.

[7]  D. Komaris,et al.  Unsupervised IMU-based evaluation of at-home exercise programmes: a feasibility study , 2022, BMC Sports Science, Medicine and Rehabilitation.

[8]  B. Vanrumste,et al.  Skeleton-Based Action Segmentation with Multi-Stage Spatial-Temporal Graph Convolutional Neural Networks , 2022, IEEE Transactions on Emerging Topics in Computing.

[9]  A. Nieuwboer,et al.  Dual Task Turning in Place: A Reliable, Valid, and Responsive Outcome Measure of Freezing of Gait , 2021, Movement disorders : official journal of the Movement Disorder Society.

[10]  M. Miatton,et al.  An Auditory Stroop Test to Implement in Cognitive Hearing Sciences: Development and Normative Data , 2021, International journal of psychological research.

[11]  S. Delp,et al.  Assessing inertial measurement unit locations for freezing of gait detection and patient preference , 2021, medRxiv.

[12]  W. L. Woo,et al.  Wearable Inertial Gait Algorithms: Impact of Wear Location and Environment in Healthy and Parkinson’s Populations , 2021, Sensors.

[13]  S. Han,et al.  The effect of levodopa on bilateral coordination and gait asymmetry in Parkinson’s disease using inertial sensor , 2021, NPJ Parkinson's disease.

[14]  L. Hadjileontiadis,et al.  DeepFoG: An IMU-Based Detection of Freezing of Gait Episodes in Parkinson’s Disease Patients via Deep Learning , 2021, Frontiers in Robotics and AI.

[15]  B. Vanrumste,et al.  Automated freezing of gait assessment with marker-based motion capture and multi-stage spatial-temporal graph convolutional neural networks , 2021, Journal of NeuroEngineering and Rehabilitation.

[16]  Edward D. Lemaire,et al.  Early Detection of Freezing of Gait during Walking Using Inertial Measurement Unit and Plantar Pressure Distribution Data , 2021, Sensors.

[17]  Vrutangkumar V. Shah,et al.  Measuring freezing of gait during daily-life: an open-source, wearable sensors approach , 2021, Journal of neuroengineering and rehabilitation.

[18]  Angelo Antonini,et al.  Foot Pressure Wearable Sensors for Freezing of Gait Detection in Parkinson’s Disease , 2020, Sensors.

[19]  E. Warmerdam,et al.  Validation of IMU-based gait event detection during curved walking and turning in older adults and Parkinson’s Disease patients , 2020, Journal of neuroengineering and rehabilitation.

[20]  T. Kahraman,et al.  Timed Up and Go Test With a Cognitive Task: Correlations With Neuropsychological Measures in People With Parkinson’s Disease , 2020, Cureus.

[21]  Kazuma Yamamoto,et al.  Resolving Class Imbalance in Object Detection with Weighted Cross Entropy Losses , 2020, ArXiv.

[22]  Shaohui Mei,et al.  Vision-Based Freezing of Gait Detection With Anatomic Directed Graph Representation , 2020, IEEE Journal of Biomedical and Health Informatics.

[23]  M. Mancini,et al.  The New Freezing of Gait Questionnaire: Unsuitable as an Outcome in Clinical Trials? , 2020, Movement disorders clinical practice.

[24]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[25]  Jorik Nonnekes,et al.  Clinical and methodological challenges for assessing freezing of gait: Future perspectives , 2019, Movement disorders : official journal of the Movement Disorder Society.

[26]  Jiabao Zhao,et al.  Temporal Convolutional Networks for Anomaly Detection in Time Series , 2019, Journal of Physics: Conference Series.

[27]  Mark Latt,et al.  Falls in Parkinson’s Disease Subtypes: Risk Factors, Locations and Circumstances , 2019, International journal of environmental research and public health.

[28]  Sergio Consoli,et al.  Artificial Intelligence in Clinical Health Care Applications: Viewpoint , 2019, Interactive journal of medical research.

[29]  Yazan Abu Farha,et al.  MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Dario Pavllo,et al.  3D Human Pose Estimation in Video With Temporal Convolutions and Semi-Supervised Training , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Julie M. Hall,et al.  Cognitive training for freezing of gait in Parkinson’s disease: a randomized controlled trial , 2018, npj Parkinson's Disease.

[32]  Konrad P. Kording,et al.  The need to approximate the use-case in clinical machine learning , 2017, GigaScience.

[33]  F. Horak,et al.  The clinical significance of freezing while turning in Parkinson’s disease , 2017, Neuroscience.

[34]  Gregory D. Hager,et al.  Temporal Convolutional Networks for Action Segmentation and Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Bastiaan R. Bloem,et al.  The Practicalities of Assessing Freezing of Gait , 2016, Journal of Parkinson's disease.

[36]  Jorge Nocedal,et al.  On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima , 2016, ICLR.

[37]  Mandar S Jog,et al.  An update on the diagnosis and treatment of Parkinson disease , 2016, Canadian Medical Association Journal.

[38]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[39]  A. Destée,et al.  Prevalence, determinants, and effect on quality of life of freezing of gait in Parkinson disease. , 2014, JAMA neurology.

[40]  Arnaud Barré,et al.  Biomechanical ToolKit: Open-source framework to visualize and process biomechanical data , 2014, Comput. Methods Programs Biomed..

[41]  Catherine Sherrington,et al.  Three simple clinical tests to accurately predict falls in people with Parkinson's disease , 2013, Movement disorders : official journal of the Movement Disorder Society.

[42]  Fay B. Horak,et al.  Quantifying freezing of gait in Parkinson's disease during the instrumented timed up and go test , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[43]  Catherine Cho,et al.  A comparison of clinical and objective measures of freezing of gait in Parkinson's disease. , 2012, Parkinsonism & related disorders.

[44]  A. Fasano,et al.  “On” state freezing of gait in Parkinson disease , 2012, Neurology.

[45]  M. Hallett,et al.  Freezing of gait: moving forward on a mysterious clinical phenomenon , 2011, The Lancet Neurology.

[46]  Kaat Desloovere,et al.  Freezing of gait in Parkinson's disease: The impact of dual‐tasking and turning , 2010, Movement disorders : official journal of the Movement Disorder Society.

[47]  Talia Herman,et al.  Reliability of the new freezing of gait questionnaire: agreement between patients with Parkinson's disease and their carers. , 2009, Gait & posture.

[48]  E. Tolosa,et al.  Validation of the freezing of gait questionnaire in patients with Parkinson's disease , 2009, Movement disorders : official journal of the Movement Disorder Society.

[49]  J. Jankovic,et al.  Movement Disorder Society‐sponsored revision of the Unified Parkinson's Disease Rating Scale (MDS‐UPDRS): Scale presentation and clinimetric testing results , 2008, Movement disorders : official journal of the Movement Disorder Society.

[50]  N. P. Quinn,et al.  The Factors that Induce or Overcome Freezing of Gait in Parkinson’s Disease , 2008, Behavioural neurology.

[51]  G. Halliday,et al.  The Sydney multicenter study of Parkinson's disease: The inevitability of dementia at 20 years , 2008, Movement disorders : official journal of the Movement Disorder Society.

[52]  W. Ondo,et al.  Ambulatory monitoring of freezing of gait in Parkinson's disease , 2008, Journal of Neuroscience Methods.

[53]  Nir Giladi,et al.  Quality of life and gender identity in Parkinson’s disease , 2005, Journal of Neural Transmission.

[54]  Nir Giladi,et al.  Characterization of freezing of gait subtypes and the response of each to levodopa in Parkinson's disease , 2003, European journal of neurology.

[55]  Nathalie Japkowicz,et al.  The class imbalance problem: A systematic study , 2002, Intell. Data Anal..

[56]  D. Altman,et al.  STATISTICAL METHODS FOR ASSESSING AGREEMENT BETWEEN TWO METHODS OF CLINICAL MEASUREMENT , 1986, The Lancet.

[57]  S. Folstein,et al.  "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. , 1975, Journal of psychiatric research.

[58]  M. Hoehn,et al.  Parkinsonism , 1967, Neurology.

[59]  S. Shapiro,et al.  An Analysis of Variance Test for Normality (Complete Samples) , 1965 .

[60]  Student,et al.  THE PROBABLE ERROR OF A MEAN , 1908 .

[61]  Liang Wang,et al.  Graph Sequence Recurrent Neural Network for Vision-Based Freezing of Gait Detection , 2020, IEEE Transactions on Image Processing.

[62]  Moran Gilat,et al.  How to Annotate Freezing of Gait from Video: A Standardized Method Using Open-Source Software. , 2019, Journal of Parkinson's disease.

[63]  M. Rudzińska,et al.  Causes and consequences of falls in Parkinson disease patients in a prospective study. , 2013, Neurologia i neurochirurgia polska.

[64]  S T Moore,et al.  Assessing the utility of Freezing of Gait Questionnaires in Parkinson's Disease. , 2012, Parkinsonism & related disorders.

[65]  E. Protas,et al.  Effects of levodopa on forward and backward gait patterns in persons with Parkinson's disease. , 2011, NeuroRehabilitation.

[66]  Marten Munneke,et al.  Clinimetrics of freezing of gait , 2008, Movement disorders : official journal of the Movement Disorder Society.

[67]  Y H Chan,et al.  Biostatistics 104: correlational analysis. , 2003, Singapore medical journal.

[68]  K. McGraw,et al.  Forming inferences about some intraclass correlation coefficients. , 1996 .

[69]  H. Levene Robust tests for equality of variances , 1961 .