Real-time Facial Expression Recognition on Robot for Healthcare

Facial expression is important indicator to human health status. This paper is devoted to improving the safety monitor and healthcare for the old through facial expression recognition (FER). A novel convolutional neural network (CNN) architecture which is able to accelerate training process was proposed to deal with FER problems. In order to improve the FER performance in real life, a new dataset was collected to ease the data imbalance of FER2013 dataset. Further more, the method of moving average is ultilized to make up for the drawbacks of still image-based approaches, which is efficient for smoothing the real-time FER results. To monitor the safety and health status of the old, a digital healthcare (DHc) framework was proposed for better healthcare. As a result, the proposed model achieved compariable performance to the state-of-the-art methods both on FER2013 and NVIE datasets. The robot, equipped with the DHc framework, can recognize facial expression in real-life with high performance, and achieved substantial improvement for safety monitor and healthcare of the old.

[1]  Cha Zhang,et al.  Image based Static Facial Expression Recognition with Multiple Deep Network Learning , 2015, ICMI.

[2]  Fei Chen,et al.  A Natural Visible and Infrared Facial Expression Database for Expression Recognition and Emotion Inference , 2010, IEEE Transactions on Multimedia.

[3]  Ahmed Ghoneim,et al.  A Facial-Expression Monitoring System for Improved Healthcare in Smart Cities , 2017, IEEE Access.

[4]  Mohammad H. Mahoor,et al.  Going deeper in facial expression recognition using deep neural networks , 2015, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[5]  K. Scherer,et al.  Introducing the Geneva Multimodal Emotion Portrayal (GEMEP) corpus , 2010 .

[6]  Frances Stewart,et al.  Providing End-of-Life Care in Care Homes for Older People , 2013, Journal of applied gerontology : the official journal of the Southern Gerontological Society.

[7]  Soo-Young Lee,et al.  Hierarchical committee of deep convolutional neural networks for robust facial expression recognition , 2016, Journal on Multimodal User Interfaces.

[8]  Stefan Wermter,et al.  Emotional expression recognition with a cross-channel convolutional neural network for human-robot interaction , 2015, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).

[9]  Afshin Dehghan,et al.  DAGER: Deep Age, Gender and Emotion Recognition Using Convolutional Neural Network , 2017, ArXiv.

[10]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[11]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[12]  Jun-Cheol Park,et al.  A Real-time Facial Expression Recognizer using Deep Neural Network , 2016, IMCOM.

[13]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Gwen Littlewort,et al.  Automatic Recognition of Facial Actions in Spontaneous Expressions , 2006, J. Multim..

[15]  Xiaoou Tang,et al.  Learning Social Relation Traits from Face Images , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[16]  John W Ragsdale,et al.  Recognizing Patients’ Emotions: Teaching Health Care Providers to Interpret Facial Expressions , 2016, Academic medicine : journal of the Association of American Medical Colleges.

[17]  Li Dan,et al.  Cognitive emotion model for eldercare robot in smart home , 2015, China Communications.

[18]  S. Herpertz,et al.  Oxytocin Promotes Facial Emotion Recognition and Amygdala Reactivity in Adults with Asperger Syndrome , 2014, Neuropsychopharmacology.

[19]  Soo-Young Lee,et al.  Fusing Aligned and Non-aligned Face Information for Automatic Affect Recognition in the Wild: A Deep Learning Approach , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[20]  Martin Kampel,et al.  Facial Expression Recognition using Convolutional Neural Networks: State of the Art , 2016, ArXiv.

[21]  Michel Valstar,et al.  Advances, Challenges, and Opportunities in Automatic Facial Expression Recognition , 2016 .

[22]  Rama Chellappa,et al.  FaceNet2ExpNet: Regularizing a Deep Face Recognition Net for Expression Recognition , 2016, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[23]  Stefan Winkler,et al.  Deep Learning for Emotion Recognition on Small Datasets using Transfer Learning , 2015, ICMI.