Improving Automatic Pain Level Recognition using Pain Site as an Auxiliary Task

Pain is an unpleasant sensory and distressing feeling usually induced by physical damages, and the intensity is further modulated by the experienced pain site. Objective assessment of pain is critical in a variety of clinical practices, however, the status quo in medical practices is based solely on self-report. Recent advancements have been observed in automatic assessment of pain using audio-video recordings, but most do not consider the complex clinical dependency between pain level and pain site. In this study, we propose a Task Specific Encoder with Soft Layer Ordering structure (TSEN-SLO) that utilizes a learnable tensor to flexibly share information between pain level and pain site while still keeping the representations of each task in their self-encoding layers to improve pain level recognition. Our network learns from both face and voice data and achieves accuracy of 70% and 48.1% in a binary and ternary self-report pain level classification in a challenging in-the-wild setting. The approach improves a relative of 6.5% and 9.1% compare to previous work on the same dataset. Further analysis also demonstrates the variation in the self-reported pain level as observed in the facial and acoustic features for different pain sites, which points toward a potential relationship between the neural-mechanism behind internal pain sensation and its effect on expressive facial/vocal behaviors.

[1]  Sridha Sridharan,et al.  Automatically Detecting Pain in Video Through Facial Action Units , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[2]  F. Cerveró Visceral versus Somatic Pain: Similarities and Differences , 2010, Digestive Diseases.

[3]  Chi-Chun Lee,et al.  Learning Conditional Acoustic Latent Representation with Gender and Age Attributes for Automatic Pain Level Recognition , 2018, INTERSPEECH.

[4]  Kerstin Eriksson,et al.  Numeric rating scale: patients' perceptions of its use in postoperative pain assessments. , 2014, Applied nursing research : ANR.

[5]  Björn Schuller,et al.  Opensmile: the munich versatile and fast open-source audio feature extractor , 2010, ACM Multimedia.

[6]  A. Dickenson,et al.  Visceral pain: the ins and outs, the ups and downs , 2012, Current opinion in supportive and palliative care.

[7]  K. Scherer,et al.  On the Acoustics of Emotion in Audio: What Speech, Music, and Sound have in Common , 2013, Front. Psychol..

[8]  Nicolai Marquardt,et al.  How Can Affect Be Detected and Represented in Technological Support for Physical Rehabilitation? , 2019, ACM Trans. Comput. Hum. Interact..

[9]  A. Stubhaug,et al.  Assessment of pain. , 2008, British journal of anaesthesia.

[10]  Maja Pantic,et al.  Continuous Pain Intensity Estimation from Facial Expressions , 2012, ISVC.

[11]  Peter Robinson,et al.  Constrained Local Neural Fields for Robust Facial Landmark Detection in the Wild , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[12]  Luis Enrique Sucar,et al.  Unobtrusive Inference of Affective States in Virtual Rehabilitation from Upper Limb Motions: A Feasibility Study , 2020, IEEE Transactions on Affective Computing.

[13]  S. Law,et al.  Assessment of acute and chronic pain , 2016, Anaesthesia & Intensive Care Medicine.

[14]  R MELZACK,et al.  The perception of pain. , 1961, Scientific American.

[15]  Elliot Meyerson,et al.  Beyond Shared Hierarchies: Deep Multitask Learning through Soft Layer Ordering , 2017, ICLR.

[16]  D. Taverner Muscle Spasm as a Cause of Somatic Pain * , 1954, Annals of the rheumatic diseases.

[17]  Albert Ali Salah,et al.  Fisher vectors with cascaded normalization for paralinguistic analysis , 2015, INTERSPEECH.

[18]  Tsuhan Chen,et al.  The painful face - Pain expression recognition using active appearance models , 2009, Image Vis. Comput..

[19]  Chi-Chun Lee,et al.  Toward Development and Evaluation of Pain Level-Rating Scale for Emergency Triage based on Vocal Characteristics and Facial Expressions , 2016, INTERSPEECH.

[20]  Chourouk Mzahi Reshaping the Village Test for investigating service brand attachment , 2014 .

[21]  K. Sluka Stimulation of Deep Somatic Tissue with Capsaicin Produces Long-Lasting Mechanical Allodynia and Heat Hypoalgesia that Depends on Early Activation of the cAMP Pathway , 2002, The Journal of Neuroscience.

[22]  Sebastian Ruder,et al.  An Overview of Multi-Task Learning in Deep Neural Networks , 2017, ArXiv.

[23]  Fabio Valente,et al.  The INTERSPEECH 2013 computational paralinguistics challenge: social signals, conflict, emotion, autism , 2013, INTERSPEECH.

[24]  Chi-Chun Lee,et al.  Embedding stacked bottleneck vocal features in a LSTM architecture for automatic pain level classification during emergency triage , 2017, 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII).

[25]  Richard H. Hunter,et al.  Abdominal Pain , 1935, The Ulster medical journal.

[26]  Thomas Mensink,et al.  Improving the Fisher Kernel for Large-Scale Image Classification , 2010, ECCV.

[27]  P. Mantyh The neurobiology of skeletal pain , 2014, The European journal of neuroscience.