Facial Expression Recognition and Positive Emotion Incentive System for Human-Robot Interaction

Facial Expression Recognition (FER) is a hot research topic currently, many efforts have been made on improving the recognition accuracy on certain datasets. Nevertheless, most of the existing works on FER are focused on verifying their algorithms on testing set, ignoring the practicability of their model in the real world. In this paper, more attention is addressed on improving the FER performance in the wild and the application of the FER model on robots. Firstly, a FER dataset is collected for training the model of facial expression recognition in the wild (FERW). Furthermore, a real-time positive emotion incentive system (PEIS) is developed for improving user experience of the robot. The proposed PEIS, which can recognize, record, analysis the emotion status of the users and give humanized feedback, consists of emotion recognition, emotion analysis and emotion feedback. Emotion recognition, the first as well as the most important part of this system, is realized by FERW based on deep learning and voting method. The PEIS is evaluated in two scenario, one is the accuracy of FERW in natural scene, and the other is the user experience of the robot employs the PEIS. Finally, experiments show that our FERW model can recognize facial expressions in real-life with an accuracy of 79%, which is practicable in the real world. Our robot XiaoBao, equipped with the PEIS, is able to enhance user experience.

[1]  Li Dan,et al.  Cognitive emotion model for eldercare robot in smart home , 2015, China Communications.

[2]  Jun-Cheol Park,et al.  A Real-time Facial Expression Recognizer using Deep Neural Network , 2016, IMCOM.

[3]  Azar Fazel,et al.  Convolutional Neural Networks for Facial Expression Recognition , 2017, ArXiv.

[4]  Dong Yu,et al.  Speech emotion recognition using deep neural network and extreme learning machine , 2014, INTERSPEECH.

[5]  Karthikeyan Shanmugasundaram,et al.  FAREC — CNN based efficient face recognition technique using Dlib , 2016, 2016 International Conference on Advanced Communication Control and Computing Technologies (ICACCCT).

[6]  Stefan Wermter,et al.  Emotional expression recognition with a cross-channel convolutional neural network for human-robot interaction , 2015, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).

[7]  Jing Xiong,et al.  A Body Emotion-Based Human-Robot Interaction , 2017, ICVS.

[8]  K. Scherer,et al.  Emotion recognition from expressions in face, voice, and body: the Multimodal Emotion Recognition Test (MERT). , 2009, Emotion.

[9]  Chengxin Li,et al.  Speech emotion recognition with acoustic and lexical features , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[10]  Yoshua Bengio,et al.  Challenges in representation learning: A report on three machine learning contests , 2013, Neural Networks.

[11]  Cha Zhang,et al.  Image based Static Facial Expression Recognition with Multiple Deep Network Learning , 2015, ICMI.

[12]  Afshin Dehghan,et al.  DAGER: Deep Age, Gender and Emotion Recognition Using Convolutional Neural Network , 2017, ArXiv.

[13]  Mohammad H. Mahoor,et al.  Going deeper in facial expression recognition using deep neural networks , 2015, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[14]  Athanasios Katsamanis,et al.  Tracking changes in continuous emotion states using body language and prosodic cues , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[15]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[16]  Hichem Sahli,et al.  Adaptive Real-Time Emotion Recognition from Body Movements , 2016, TIIS.

[17]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Ning An,et al.  Speech Emotion Recognition Using Fourier Parameters , 2015, IEEE Transactions on Affective Computing.

[19]  Javier G. Rázuri,et al.  Speech emotion recognition in emotional feedback for Human-Robot Interaction , 2015 .

[20]  Martin Kampel,et al.  Facial Expression Recognition using Convolutional Neural Networks: State of the Art , 2016, ArXiv.