Pain Assessment From Facial Expression: Neonatal Convolutional Neural Network (N-CNN)

The current standard for assessing neonatal pain is discontinuous and suffers from inter-observer variations, which can result in delayed intervention and inconsistent treatment of pain. Therefore, it is critical to address the shortcomings of the current standard and develop continuous and less subjective pain assessment tools. Convolutional Neural Networks have gained much popularity in the last decades due to the wide range of its successful applications in medical image analysis, object recognition, and emotion recognition. In this paper, we propose a Neonatal Convolutional Neural Network, designed and trained end-to-end to detect neonatal pain. We evaluated the proposed network in two data sets of neonates and compared its performance to the performance of ResNet architecture in the same data sets. Our proposed method outperformed ResNet in recognizing neonates’ pain and achieved around 91.00% accuracy. While further research is needed, our preliminary results suggest that the presented network can be used for automatic pain assessment, and possibly similar applications. It also suggests that the automatic recognition of neonatal pain provides a viable and more efficient alternative to the current standard of pain assessment.

[1]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[2]  Corneliu Florea,et al.  Learning Pain from Emotion: Transferred HoT Data Representation for Pain Intensity Estimation , 2014, ECCV Workshops.

[3]  Yu Sun,et al.  Pain assessment in infants: Towards spotting pain expression based on infants' facial strain , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[4]  Takeo Kanade,et al.  Dense 3D face alignment from 2D videos in real-time , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[5]  S. Mackey,et al.  Towards a Physiology-Based Measure of Pain: Patterns of Human Brain Activity Distinguish Painful from Non-Painful Thermal Stimulation , 2011, PloS one.

[6]  Sheryl Brahnam,et al.  Machine assessment of neonatal facial expressions of acute pain , 2007, Decis. Support Syst..

[7]  Ayoub Al-Hamadi,et al.  Comparative Learning Applied to Intensity Rating of Facial Expressions of pain , 2014, Int. J. Pattern Recognit. Artif. Intell..

[8]  M. Ranger,et al.  Innovating in pain assessment of the critically ill: exploring cerebral near-infrared spectroscopy as a bedside approach. , 2014, Pain management nursing : official journal of the American Society of Pain Management Nurses.

[9]  Yu Sun,et al.  Infants' Pain Recognition Based on Facial Expression: Dynamic Hybrid Descriptions , 2018, IEICE Trans. Inf. Syst..

[10]  H. Traue,et al.  Pain Intensity Recognition Rates via Biopotential Feature Patterns with Support Vector Machines , 2015, PloS one.

[11]  T. Voepel-Lewis,et al.  The Reliability and Validity of the Face, Legs, Activity, Cry, Consolability Observational Tool as a Measure of Pain in Children with Cognitive Impairment , 2002, Anesthesia and analgesia.

[12]  Luigi Celona,et al.  Neonatal Facial Pain Assessment Combining Hand-Crafted and Deep Features , 2017, ICIAP Workshops.

[13]  Nikhil Ketkar,et al.  Introduction to Keras , 2017 .

[14]  Liisa Holsti,et al.  Specific Newborn Individualized Developmental Care and Assessment Program movements are associated with acute pain in preterm infants in the neonatal intensive care unit. , 2004, Pediatrics.

[15]  Peter H. N. de With,et al.  Video-based discomfort detection for infants , 2018, Machine Vision and Applications.

[16]  Andrew Shea,et al.  DeepFaceLIFT: Interpretable Personalized Models for Automatic Estimation of Self-Reported Pain , 2017, AffComp@IJCAI.

[17]  Matthew S. Goodwin,et al.  Automated Assessment of Children’s Postoperative Pain Using Computer Vision , 2015, Pediatrics.

[18]  Yu Sun,et al.  Machine-based Multimodal Pain Assessment Tool for Infants: A Review , 2016, ArXiv.

[19]  D. Tibboel,et al.  Neonatal Facial Coding System for Assessing Postoperative Pain in Infants: Item Reduction is Valid and Feasible , 2003, The Clinical journal of pain.

[20]  L. Storme,et al.  Newborn Infant Pain Assessment Using Heart Rate Variability Analysis , 2010, The Clinical journal of pain.

[21]  Samuel H. Hawkins,et al.  Predicting malignant nodules by fusing deep features with classical radiomics features , 2018, Journal of medical imaging.

[22]  Rangachar Kasturi,et al.  A Review of Automated Pain Assessment in Infants: Features, Classification Tasks, and Databases , 2018, IEEE Reviews in Biomedical Engineering.

[23]  Andrew Zisserman,et al.  Return of the Devil in the Details: Delving Deep into Convolutional Nets , 2014, BMVC.

[24]  M. Cruz,et al.  Epidemiology of painful procedures performed in neonates: A systematic review of observational studies , 2016, European journal of pain.

[25]  K. Craig,et al.  Judging pain in infants: behavioural, contextual, and developmental determinants , 1997, Pain.

[26]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[27]  Yu Sun,et al.  Automated Pain Assessment in Neonates , 2017, SCIA.

[28]  Marco Dondi,et al.  Facial Action Coding System for Infants and Young Children (Baby FACS) , 2016 .

[29]  Rosalind W. Picard,et al.  338) Automatic detection of nociceptive stimuli and pain intensity from facial expressions , 2017 .

[30]  Dmitry B. Goldgof,et al.  An approach for automated multimodal analysis of infants' pain , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[31]  Peter H. N. de With,et al.  Video-based facial discomfort analysis for infants , 2014, Electronic Imaging.

[32]  Loris Nanni,et al.  A local approach based on a Local Binary Patterns variant texture descriptor for classifying pain states , 2010, Expert Syst. Appl..

[33]  D. Hudson-Barr,et al.  Validation of the Pain Assessment in Neonates (PAIN) Scale with the Neonatal Infant Pain Scale (NIPS) , 2002, Neonatal Network.

[34]  Neeru Rathee,et al.  Multiview Distance Metric Learning on facial feature descriptors for automatic pain intensity detection , 2016, Comput. Vis. Image Underst..

[35]  Kamal Nasrollahi,et al.  Deep Pain: Exploiting Long Short-Term Memory Networks for Facial Expression Classification , 2017, IEEE Transactions on Cybernetics.

[36]  A. Ng Feature selection, L1 vs. L2 regularization, and rotational invariance , 2004, Twenty-first international conference on Machine learning - ICML '04.

[37]  Gang Hua,et al.  A convolutional neural network cascade for face detection , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[40]  J. Peters,et al.  Long-term consequences of pain in human neonates. , 2006, Seminars in fetal & neonatal medicine.

[41]  Chih-Yun Pai Automatic Pain Assessment from Infants’ Crying Sounds , 2016 .

[42]  Terri Ashmeade,et al.  Automatic Infants’ Pain Assessment by Dynamic Facial Representation: Effects of Profile View, Gestational Age, Gender, and Race , 2018, Journal of clinical medicine.

[43]  T. Field Preterm newborn pain research review. , 2017, Infant behavior & development.

[44]  Yu Sun,et al.  Neonatal Pain Expression Recognition Using Transfer Learning , 2018, ArXiv.