Pose and Optical Flow Fusion (POFF) for accurate tremor detection and quantification

Abstract Limb tremor measurements are one factor used to characterize and quantify the severity of neurodegenerative disorders. These tremor measurements can also provide dosage-response feedback to guide medication treatments. Here, we propose a system to automatically measure limb tremors in home or clinic settings. The key feature of proposed method is that it is contactless; not requiring a user to wear or hold a device or marker. Our sensor is a Kinect 2, which measures color and depth and estimates rough limb motion. We show that its pose accuracy is poor for small limb tremors below 10 mm amplitude, and so we propose an additional level of tremor tracking that recovers limb motion at a higher precision. Our method upgrades the sensitivity to achieve detection and analysis for tremors down to 2 mm amplitude. We include empirical experiments and measurements showing improved tremor amplitude and frequency estimation using our proposed Pose and Optical Flow Fusion (POFF) algorithm.

[1]  Alberto Olivares,et al.  Wagyromag: Wireless sensor network for monitoring and processing human body movement in healthcare applications , 2011, J. Syst. Archit..

[2]  Susan M. Astley,et al.  Evaluation of Kinect 3D Sensor for Healthcare Imaging , 2016, Journal of medical and biological engineering.

[3]  Tipu Z. Aziz,et al.  Resting tremor classification and detection in Parkinson's disease patients , 2015, Biomed. Signal Process. Control..

[4]  K. Lewenstein,et al.  Verification of the functionality of device for monitoring human tremor , 2015 .

[5]  Rubén Posada-Gómez,et al.  A Brief Review on the Validity and Reliability of Microsoft Kinect Sensors for Functional Assessment Applications , 2018 .

[6]  Livio Pinto,et al.  Calibration of Kinect for Xbox One and Comparison between the Two Generations of Microsoft Sensors , 2015, Sensors.

[7]  Deepak Joshi,et al.  An automatic non-invasive method for Parkinson's disease classification , 2017, Comput. Methods Programs Biomed..

[8]  Garrett M. Clayton,et al.  Integrating the Microsoft Kinect With Simulink: Real-Time Object Tracking Example , 2014, IEEE/ASME Transactions on Mechatronics.

[9]  José Luis Pons Rovira,et al.  Real-Time Estimation of Pathological Tremor Parameters from Gyroscope Data , 2010, Sensors.

[10]  Takeshi Takaki,et al.  Color-histogram-based tracking at 2000 fps , 2012, J. Electronic Imaging.

[11]  A. Kääb,et al.  Sub-pixel precision image matching for measuring surface displacements on mass movements using normalized cross-correlation , 2011 .

[12]  F. Cavallo,et al.  How Wearable Sensors Can Support Parkinson's Disease Diagnosis and Treatment: A Systematic Review , 2017, Front. Neurosci..

[13]  Jinxiang Chai,et al.  Accurate realtime full-body motion capture using a single depth camera , 2012, ACM Trans. Graph..

[14]  K. Norman,et al.  A randomized controlled trial of the effects of weights on amplitude and frequency of postural hand tremor in people with Parkinson's disease , 2002, Clinical rehabilitation.

[15]  G. Cosoli,et al.  Non-contact measurement of tremor for the characterisation of Parkinsonian individuals: comparison between Kinect and Laser Doppler vibrometer , 2017 .

[16]  Kamiar Aminian,et al.  Quantification of Tremor and Bradykinesia in Parkinson's Disease Using a Novel Ambulatory Monitoring System , 2007, IEEE Transactions on Biomedical Engineering.

[17]  Wei Cai,et al.  A Kinect(™) camera based navigation system for percutaneous abdominal puncture. , 2016, Physics in medicine and biology.

[18]  J W Langston,et al.  Quantification of dyskinesia in Parkinson's disease: Validation of a novel instrumental method , 1999, Movement disorders : official journal of the Movement Disorder Society.

[19]  Çağatay Berke Erdaş,et al.  Parkinson's disease monitoring from gait analysis via foot-worn sensors , 2018 .

[20]  Tim Lüth,et al.  Quantitative Assessment of Parkinsonian Tremor Based on an Inertial Measurement Unit , 2015, Sensors.

[21]  P. Olivier,et al.  Accuracy of the Microsoft Kinect sensor for measuring movement in people with Parkinson's disease. , 2014, Gait & posture.

[22]  Zhaoying Zhou,et al.  A real-time articulated human motion tracking using tri-axis inertial/magnetic sensors package. , 2004, IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[23]  R. Fisher THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[24]  Debeshi Dutta,et al.  Bayesian network aided grasp and grip efficiency estimation using a smart data glove for post-stroke diagnosis , 2017 .

[25]  Dieter Fox,et al.  RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments , 2012, Int. J. Robotics Res..

[26]  C. Chern,et al.  Detection of hand tremor in patients with Parkinson’s disease using a non-invasive laser line triangulation measurement method , 2016 .

[27]  Frank Mayer,et al.  Reliability and validity of the Kinect V2 for the assessment of lower extremity rehabilitation exercises. , 2019, Gait & posture.

[28]  Dose‐Response Analysis of the Effect of Carbidopa‐Levodopa Extended‐Release Capsules (IPX066) in Levodopa‐Naive Patients With Parkinson Disease , 2016, Journal of clinical pharmacology.

[29]  W. Grooten,et al.  Reliability and validity of a novel Kinect-based software program for measuring posture, balance and side-bending , 2018, BMC Musculoskeletal Disorders.

[30]  Jyh-Cheng Chen,et al.  A Parkinson’s Disease Measurement System Using Laser Lines and a CMOS Image Sensor , 2011, Sensors.

[31]  Oliver M O'Reilly,et al.  A Kinect-based movement assessment system: marker position comparison to Vicon , 2017, Computer methods in biomechanics and biomedical engineering.

[32]  Anne Beuter,et al.  Effect of deep brain stimulation on amplitude and frequency characteristics of rest tremor in Parkinson’s disease , 2001 .

[33]  Xu Xu,et al.  Accuracy of the Microsoft Kinect for measuring gait parameters during treadmill walking. , 2015, Gait & posture.

[34]  G. Deuschl,et al.  The pathophysiology of tremor , 2001, Muscle & nerve.

[35]  Slavka Viteckova,et al.  Motion Capture System for Finger Movement Measurement in Parkinson Disease , 2014 .

[36]  C. Marsden,et al.  Physiological and pathological tremors and rhythmic central motor control. , 2000, Brain : a journal of neurology.

[37]  M. Brin,et al.  Consensus Statement of the Movement Disorder Society on Tremor , 2008, Movement disorders : official journal of the Movement Disorder Society.

[38]  Eduardo Rocon,et al.  Identification of activities of daily living in tremorous patients using inertial sensors , 2017, Expert Syst. Appl..

[39]  Pedro Arias,et al.  Metrological comparison between Kinect I and Kinect II sensors , 2015 .

[40]  M J Rosen,et al.  A wearable tremor-suppression orthosis. , 1998, Journal of rehabilitation research and development.

[41]  Toby Sharp,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR.

[42]  Kathleen A. Bieryla Xbox Kinect training to improve clinical measures of balance in older adults: a pilot study , 2016, Aging Clinical and Experimental Research.

[43]  J P Cobb,et al.  Validity and sensitivity of the longitudinal asymmetry index to detect gait asymmetry using Microsoft Kinect data. , 2017, Gait & posture.

[44]  Raymond W. McGorry,et al.  The validity of the first and second generation Microsoft Kinect™ for identifying joint center locations during static postures. , 2015, Applied ergonomics.

[45]  Haibo Li,et al.  Direct three-dimensional head pose estimation from Kinect-type sensors , 2014 .

[46]  Daniel M Wolpert,et al.  Sensory attenuation in Parkinson’s disease is related to disease severity and dopamine dose , 2018, Scientific Reports.

[47]  Kai-Hsiang Chen,et al.  Development of method for quantifying essential tremor using a small optical device , 2016, Journal of Neuroscience Methods.

[48]  Victor Sholukha,et al.  3D Analysis of Upper Limbs Motion during Rehabilitation Exercises Using the KinectTM Sensor: Development, Laboratory Validation and Clinical Application , 2018, Sensors.

[49]  Yu-Wing Tai,et al.  Accurate and real-time depth video acquisition using Kinect–stereo camera fusion , 2014 .

[50]  Hong Wang,et al.  Motion trajectory of human arms based on the dual quaternion with motion tracker , 2015, Multimedia Tools and Applications.

[51]  Ke Yang,et al.  Objective and quantitative assessment of motor function in Parkinson's disease-from the perspective of practical applications. , 2016, Annals of translational medicine.

[52]  M Gresty,et al.  Frequency/amplitude characteristics of postural tremor of the hands in a population of patients with bilateral essential tremor: implications for the classification and mechanism of essential tremor. , 1987, Journal of neurology, neurosurgery, and psychiatry.

[53]  Jenq-Neng Hwang,et al.  Tremor detection using motion filtering and SVM , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[54]  Kevin Warwick,et al.  Parkinsonian tremor identification with multiple local field potential feature classification , 2012, Journal of Neuroscience Methods.

[55]  Marjorie Skubic,et al.  Fall Detection in Homes of Older Adults Using the Microsoft Kinect , 2015, IEEE Journal of Biomedical and Health Informatics.

[56]  Chase Haddix,et al.  Upper extremity movement reliability and validity of the Kinect version 2 , 2018, Disability and rehabilitation. Assistive technology.

[57]  Boris Rubinsky,et al.  Noncontact Tremor Characterization Using Low-Power Wideband Radar Technology , 2012, IEEE Transactions on Biomedical Engineering.

[58]  Judith E Deutsch,et al.  Validity and Reliability of the Kinect for Assessment of Standardized Transitional Movements and Balance: Systematic Review and Translation into Practice. , 2019, Physical medicine and rehabilitation clinics of North America.