Automatic Estimation of Self-Reported Pain by Interpretable Representations of Motion Dynamics

We propose an automatic method for pain intensity measurement from video. For each video, pain intensity was measured using the dynamics of facial movement using 66 facial points. Gram matrices formulation was used for facial points trajectory representations on the Riemannian manifold of symmetric positive semi-definite matrices of fixed rank. Curve fitting and temporal alignment were then used to smooth the extracted trajectories. A Support Vector Regression model was then trained to encode the extracted trajectories into ten pain intensity levels consistent with the Visual Analogue Scale for pain intensity measurement. The proposed approach was evaluated using the UNBC McMaster Shoulder Pain Archive and was compared to the state-of-the-art on the same data. Using both 5-fold cross-validation and leave-one-subject-out cross-validation, our results are competitive with respect to state-of-the-art methods.

[1]  M. Jensen,et al.  Interpretation of visual analog scale ratings and change scores: a reanalysis of two clinical trials of postoperative pain. , 2003, The journal of pain : official journal of the American Pain Society.

[2]  Silvere Bonnabel,et al.  Regression on Fixed-Rank Positive Semidefinite Matrices: A Riemannian Approach , 2010, J. Mach. Learn. Res..

[3]  Andrew Shea,et al.  DeepFaceLIFT: Interpretable Personalized Models for Automatic Estimation of Self-Reported Pain , 2017, AffComp@IJCAI.

[4]  K. Prkachin,et al.  The structure, reliability and validity of pain expression: Evidence from patients with shoulder pain , 2008, PAIN.

[5]  Bart Vandereycken,et al.  A Riemannian geometry with complete geodesics for the set of positive semidefinite matrices of fixed rank , 2013 .

[6]  P. Ekman,et al.  Facial action coding system , 2019 .

[7]  M. Kunz,et al.  The facial expression of pain in patients with dementia , 2007, Pain.

[8]  Pierre-Antoine Absil,et al.  Quotient Geometry with Simple Geodesics for the Manifold of Fixed-Rank Positive-Semidefinite Matrices , 2020, SIAM J. Matrix Anal. Appl..

[9]  Bamdev Mishra,et al.  Manopt, a matlab toolbox for optimization on manifolds , 2013, J. Mach. Learn. Res..

[10]  Jeffrey F. Cohn,et al.  Painful data: The UNBC-McMaster shoulder pain expression archive database , 2011, Face and Gesture 2011.

[11]  Pierre-Yves Gousenbourger,et al.  Data Fitting on Manifolds with Composite Bézier-Like Curves and Blended Cubic Splines , 2018, Journal of Mathematical Imaging and Vision.

[12]  Jesse A. Berlin,et al.  Defining the clinically important difference in pain outcome measures , 2000, PAIN.

[13]  Pierre-Antoine Absil,et al.  Curvature of the Manifold of Fixed-Rank Positive-Semidefinite Matrices Endowed with the Bures-Wasserstein Metric , 2019, GSI.

[14]  Laurent Jacques,et al.  Piecewise-Bézier C1 smoothing on manifolds with application to wind field estimation , 2017, ESANN.

[15]  R. Redi [FACIAL EXPRESSION IN PAIN]. , 1965, Rassegna clinico-scientifica.

[16]  H. Diener,et al.  Pain measurement: Visual Analogue Scale (VAS) and Verbal Rating Scale (VRS) in clinical trials with OTC analgesics in headache , 2012, Cephalalgia : an international journal of headache.

[17]  Alexander J. Smola,et al.  Support Vector Regression Machines , 1996, NIPS.

[18]  Jeffrey F. Cohn,et al.  Automatic, Objective, and Efficient Measurement of Pain Using Automated Face Analysis , 2018 .

[19]  H. Merskey Pain terms: a list with definitions and notes on usage. Recommended by the IASP Subcommittee on Taxonomy. , 1979, Pain.

[20]  Tomoko Matsui,et al.  A Kernel for Time Series Based on Global Alignments , 2006, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[21]  Alberto Del Bimbo,et al.  Fitting, Comparison, and Alignment of Trajectories on Positive Semi-Definite Matrices with Application to Action Recognition , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[22]  Ruth E. Grunau,et al.  The facial expression of pain. , 2001 .

[23]  M. Jensen,et al.  The meaning of pain relief in a clinical trial. , 2005, The journal of pain : official journal of the American Pain Society.

[24]  Pierre-Yves Gousenbourger,et al.  Interpolation on the manifold of fixed-rank positive-semidefinite matrices for parametric model order reduction: preliminary results , 2019, ESANN.

[25]  K. Prkachin The consistency of facial expressions of pain: a comparison across modalities , 1992, Pain.

[26]  Bart Vandereycken,et al.  Embedded geometry of the set of symmetric positive semidefinite matrices of fixed rank , 2009, 2009 IEEE/SP 15th Workshop on Statistical Signal Processing.

[27]  Francis R. Bach,et al.  Low-Rank Optimization on the Cone of Positive Semidefinite Matrices , 2008, SIAM J. Optim..

[28]  S. Mackey,et al.  Pain outcomes: A brief review of instruments and techniques , 2009, Current pain and headache reports.

[29]  Rosalind W. Picard,et al.  Automatic Recognition Methods Supporting Pain Assessment: A Survey , 2019, IEEE Transactions on Affective Computing.

[30]  Stefano Berretti,et al.  A Novel Geometric Framework on Gram Matrix Trajectories for Human Behavior Understanding , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Ognjen Rudovic,et al.  Personalized Automatic Estimation of Self-Reported Pain Intensity from Facial Expressions , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[32]  Mehrtash Tafazzoli Harandi,et al.  Image set classification by symmetric positive semi-definite matrices , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[33]  Silvere Bonnabel,et al.  Riemannian Metric and Geometric Mean for Positive Semidefinite Matrices of Fixed Rank , 2008, SIAM J. Matrix Anal. Appl..