Audio-Visual Recognition of Pain Intensity

In this work, a multi-modal pain intensity recognition system based on both audio and video channels is presented. The system is assessed on a newly recorded dataset consisting of several individuals, each subjected to 3 gradually increasing levels of painful heat stimuli under controlled conditions. The assessment of the dataset consists of the extraction of a multitude of features from each modality, followed by an evaluation of the discriminative power of each extracted feature set. Finally, several fusion architectures, involving early and late fusion, are assessed. The temporal availability of the audio channel is taken in consideration during the assessment of the fusion architectures.

[1]  Ayoub Al-Hamadi,et al.  Towards Pain Monitoring: Facial Expression, Head Pose, a new Database, an Automatic System and Remaining , 2013, BMVC.

[2]  Patrick Thiam,et al.  Adaptive confidence learning for the personalization of pain intensity estimation systems , 2017, Evol. Syst..

[3]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[4]  N. Ramesh Babu,et al.  Speech recognition using MFCC and DTW , 2014, 2014 International Conference on Advances in Electrical Engineering (ICAEE).

[5]  Xuewen Wu,et al.  Combining Multimodal Features within a Fusion Network for Emotion Recognition in the Wild , 2015, ICMI.

[6]  Nicolai Marquardt,et al.  Pain level recognition using kinematics and muscle activity for physical rehabilitation in chronic pain , 2015, 2015 International Conference on Affective Computing and Intelligent Interaction (ACII).

[7]  Andrew Zisserman,et al.  Representing shape with a spatial pyramid kernel , 2007, CIVR '07.

[8]  Maja Pantic,et al.  The Automatic Detection of Chronic Pain-Related Expression: Requirements, Challenges and the Multimodal EmoPain Dataset , 2016, IEEE Transactions on Affective Computing.

[9]  Patrick Thiam,et al.  Multimodal Data Fusion for Person-Independent, Continuous Estimation of Pain Intensity , 2015, EANN.

[10]  Matti Pietikäinen,et al.  Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Shashidhar G. Koolagudi,et al.  Emotion Recognition Using Vocal Tract Information , 2013 .

[12]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[13]  Sascha Meudt,et al.  On Instance Selection in Audio Based Emotion Recognition , 2012, ANNPR.

[14]  Yiwen Zhao,et al.  Physiological Signals Based Quantitative Evaluation Method of the Pain , 2014 .

[15]  Hynek Hermansky,et al.  RASTA-PLP speech analysis technique , 1992, [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[16]  H. Traue,et al.  Pain Intensity Recognition Rates via Biopotential Feature Patterns with Support Vector Machines , 2015, PloS one.

[17]  Markus Kächele,et al.  Using Radial Basis Function Neural Networks for Continuous and Discrete Pain Estimation from Bio-physiological Signals , 2016, ANNPR.

[18]  Corneliu Florea,et al.  Learning Pain from Emotion: Transferred HoT Data Representation for Pain Intensity Estimation , 2014, ECCV Workshops.

[19]  Patrick Thiam,et al.  Methods for Person-Centered Continuous Pain Intensity Assessment From Bio-Physiological Channels , 2016, IEEE Journal of Selected Topics in Signal Processing.

[20]  Björn W. Schuller,et al.  Recent developments in openSMILE, the munich open-source multimedia feature extractor , 2013, ACM Multimedia.

[21]  Maja Pantic,et al.  Continuous Pain Intensity Estimation from Facial Expressions , 2012, ISVC.

[22]  Peter Robinson,et al.  OpenFace: An open source facial behavior analysis toolkit , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[23]  Ayoub Al-Hamadi,et al.  Automatic Pain Recognition from Video and Biomedical Signals , 2014, 2014 22nd International Conference on Pattern Recognition.

[24]  Markus Kächele,et al.  Cascaded Fusion of Dynamic, Spatial, and Textural Feature Sets for Person-Independent Facial Emotion Recognition , 2014, 2014 22nd International Conference on Pattern Recognition.

[25]  Markus Kächele,et al.  Bio-Visual Fusion for Person-Independent Recognition of Pain Intensity , 2015, MCS.

[26]  Ludmila I. Kuncheva,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2004 .

[27]  Shashidhar G. Koolagudi,et al.  Speech Emotion Recognition: A Review , 2013 .