Hand-tremor frequency estimation in videos

We focus on the problem of estimating human hand-tremor frequency from input RGB video data. Estimating tremors from video is important for non-invasive monitoring, analyzing and diagnosing patients suffering from motor-disorders such as Parkinson’s disease. We consider two approaches for hand-tremor frequency estimation: (a) a Lagrangian approach where we detect the hand at every frame in the video, and estimate the tremor frequency along the trajectory; and (b) an Eulerian approach where we first localize the hand, we subsequently remove the large motion along the movement trajectory of the hand, and we use the video information over time encoded as intensity values or phase information to estimate the tremor frequency. We estimate hand tremors on a new human tremor dataset, TIM-Tremor, containing static tasks as well as a multitude of more dynamic tasks, involving larger motion of the hands. The dataset has 55 tremor patient recordings together with: associated ground truth accelerometer data from the most affected hand, RGB video data, and aligned depth data.

[1]  David J. Fleet,et al.  Computation of component image velocity from local phase information , 1990, International Journal of Computer Vision.

[2]  Arnold W. M. Smeulders,et al.  Real-World Repetition Estimation by Div, Grad and Curl , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[3]  Julian F. P. Kooij.  SenseCap: Synchronized Data Collection with Microsoft Kinect2 and LeapMotion , 2016, ACM Multimedia.

[4]  Silvia L. Pintea,et al.  Video Acceleration Magnification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Yuji Kokumai,et al.  Non-contact Full Field Vibration Measurement Based on Phase-Shifting , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Silvia L. Pintea,et al.  Making a Case for Learning Motion Representations with Phase , 2016, ECCV Workshops.

[7]  Jose A. Perea,et al.  (Quasi)Periodicity Quantification in Video Data, Using Topology , 2017, SIAM J. Imaging Sci..

[8]  Edward H. Adelson,et al.  The Design and Use of Steerable Filters , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Kwang Suk Park,et al.  Automatic Classification of Tremor Severity in Parkinson’s Disease Using a Wearable Device , 2017, Sensors.

[10]  Jitendra Malik,et al.  Human Pose Estimation with Iterative Error Feedback , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Julian F. P. Kooij,et al.  Depth-Aware Motion Magnification , 2016, ECCV.

[12]  Dana Kulic,et al.  Parkinson's Disease Assessment from a Wrist-Worn Wearable Sensor in Free-Living Conditions: Deep Ensemble Learning and Visualization , 2018, ArXiv.

[13]  James McNames,et al.  Using Portable Transducers to Measure Tremor Severity , 2016, Tremor and other hyperkinetic movements.

[14]  Frédo Durand,et al.  Phase-based video motion processing , 2013, ACM Trans. Graph..

[15]  Randal C. Nelson,et al.  Detection and Recognition of Periodic, Nonrigid Motion , 1997, International Journal of Computer Vision.

[16]  Varun Ramakrishna,et al.  Pose Machines: Articulated Pose Estimation via Inference Machines , 2014, ECCV.

[17]  G. Subrahmanya V. R. K. Rao,et al.  A learning based approach for tremor detection from videos , 2013, 2013 IEEE Conference on Open Systems (ICOS).

[18]  Georgios Tzimiropoulos,et al.  Human Pose Estimation via Convolutional Part Heatmap Regression , 2016, ECCV.

[19]  Peter V. Gehler,et al.  DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Václav Hlavác,et al.  TremAn: A tool for measuring tremor frequency from video sequences , 2010, Movement disorders : official journal of the Movement Disorder Society.

[21]  Bernt Schiele,et al.  2D Human Pose Estimation: New Benchmark and State of the Art Analysis , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Roman Goldenberg,et al.  Behavior classification by eigendecomposition of periodic motions , 2005, Pattern Recognit..

[23]  Michelle A. Burack,et al.  Automatic assessment of medication states of patients with Parkinson's disease using wearable sensors , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[24]  Chris D. Nugent,et al.  Validation of a new tool for automatic assessment of tremor frequency from video recordings , 2011, Journal of Neuroscience Methods.

[25]  Lior Wolf,et al.  Live Repetition Counting , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[26]  Zaidi Mohd Ripin,et al.  Pathological Hand Tremor Measurement—Challenges and Advances , 2017 .

[27]  Emmanuel Roze,et al.  A simple way to distinguish essential tremor from tremulous Parkinson's disease. , 2017, Brain : a journal of neurology.

[28]  Zhen He,et al.  Continuous Video to Simple Signals for Swimming Stroke Detection with Convolutional Neural Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[29]  John-Stuart Brittain,et al.  Tremor stability index: a new tool for differential diagnosis in tremor syndromes , 2017, Brain : a journal of neurology.

[30]  Frédo Durand,et al.  Visual vibrometry: Estimating material properties from small motions in video , 2015, CVPR.

[31]  Jenq-Neng Hwang,et al.  Tremor detection using motion filtering and SVM , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[32]  Nicola J. Ferrier,et al.  Repetitive motion analysis: segmentation and event classification , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Paul Zarchan,et al.  Fundamentals of Kalman Filtering: A Practical Approach , 2001 .

[34]  Hai Su,et al.  A new method for evaluating postural hand tremor based on CMOS camera , 2015 .

[35]  Frédo Durand,et al.  Video magnification in presence of large motions , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Tim Lüth,et al.  Quantitative Assessment of Parkinsonian Tremor Based on an Inertial Measurement Unit , 2015, Sensors.

[37]  Varun Ramakrishna,et al.  Convolutional Pose Machines , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Larry S. Davis,et al.  Pedestrian Detection via Periodic Motion Analysis , 2007, International Journal of Computer Vision.