An Emotion Recognition System for Mobile Applications

Emotion-aware mobile applications have been increasing due to their smart features and user acceptability. To realize such an application, an emotion recognition system should be in real time and highly accurate. As a mobile device has limited processing power, the algorithm in the emotion recognition system should be implemented using less computation. In this paper, we propose an emotion recognition with high performance for mobile applications. In the proposed system, facial video is captured by an embedded camera of a smart phone. Some representative frames are extracted from the video, and a face detection module is applied to extract the face regions in the frames. The Bandlet transform is realized on the face regions, and the resultant subband is divided into non-overlapping blocks. Local binary patterns’ histograms are calculated for each block, and then are concatenated over all the blocks. The Kruskal–Wallis feature selection is applied to select the most dominant bins of the concatenated histograms. The dominant bins are then fed into a Gaussian mixture model-based classifier to classify the emotion. Experimental results show that the proposed system achieves high recognition accuracy in a reasonable time.

[1]  Michael J. Lyons,et al.  Coding facial expressions with Gabor wavelets , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[2]  Min Chen,et al.  CP-Robot: Cloud-Assisted Pillow Robot for Emotion Sensing and Interaction , 2016 .

[3]  Victor C. M. Leung,et al.  EMC: Emotion-aware mobile cloud computing in 5G , 2015, IEEE Network.

[4]  Min Chen,et al.  iDoctor: Personalized and professionalized medical recommendations based on hybrid matrix factorization , 2017, Future Gener. Comput. Syst..

[5]  Oksam Chae,et al.  Robust Facial Expression Recognition Based on Local Directional Pattern , 2010 .

[6]  M. Shamim Hossain,et al.  Audio-visual emotion recognition using multi-directional regression and Ridgelet transform , 2016, Journal on Multimodal User Interfaces.

[7]  Weishan Zhang,et al.  Deep Learning Based Emotion Recognition from Chinese Speech , 2016, ICOST.

[8]  Kyandoghere Kyamakya,et al.  EEG-based emotion recognition approach for e-healthcare applications , 2016, ICUFN.

[9]  Christopher Joseph Pal,et al.  EmoNets: Multimodal deep learning approaches for emotion recognition in video , 2015, Journal on Multimodal User Interfaces.

[10]  José Manuel Pastor,et al.  Software Architecture for Smart Emotion Recognition and Regulation of the Ageing Adult , 2016, Cognitive Computation.

[11]  Yin Zhang,et al.  GroRec: A Group-Centric Intelligent Recommender System Integrating Social, Mobile and Big Data Technologies , 2016, IEEE Transactions on Services Computing.

[12]  Cecilia Mascolo,et al.  EmotionSense: a mobile phones based adaptive platform for experimental social psychology research , 2010, UbiComp.

[13]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  M. Shamim Hossain,et al.  Audio-Visual Emotion Recognition Using Big Data Towards 5G , 2016, Mob. Networks Appl..

[15]  Muhammad Ghulam,et al.  User emotion recognition from a larger pool of social network data using active learning , 2017, Multimedia Tools and Applications.

[16]  Min Chen,et al.  AIWAC: affective interaction through wearable computing and cloud technology , 2015, IEEE Wireless Communications.

[17]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[18]  John W Ragsdale,et al.  Recognizing Patients’ Emotions: Teaching Health Care Providers to Interpret Facial Expressions , 2016, Academic medicine : journal of the Association of American Medical Colleges.

[19]  Raphael C.-W. Phan,et al.  Facial Expression Recognition in the Encrypted Domain Based on Local Fisher Discriminant Analysis , 2013, IEEE Transactions on Affective Computing.

[20]  Stéphane Mallat,et al.  A review of Bandlet methods for geometrical image representation , 2007, Numerical Algorithms.

[21]  Adam Wierzbicki,et al.  Emotion Aware Mobile Application , 2010, ICCCI.

[22]  Shiqing Zhang,et al.  Facial Expression Recognition Based on Local Binary Patterns and Kernel Discriminant Isomap , 2011, Sensors.

[23]  Min Chen,et al.  Demo: LIVES: Learning through Interactive Video and Emotion-aware System , 2015, MobiHoc.

[24]  Victor C. M. Leung,et al.  CAP: community activity prediction based on big data analysis , 2014, IEEE Network.

[25]  Hyo Jong Lee,et al.  A novel feature extraction method based on late positive potential for emotion recognition in human brain signal patterns , 2016, Comput. Electr. Eng..

[26]  Kyandoghere Kyamakya,et al.  EEG-based emotion recognition approach for e-healthcare applications , 2016, 2016 Eighth International Conference on Ubiquitous and Future Networks (ICUFN).

[27]  M. Shamim Hossain,et al.  Audio–Visual Emotion-Aware Cloud Gaming Framework , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[28]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[29]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[30]  Ya Li,et al.  Multi-scale Temporal Modeling for Dimensional Emotion Recognition in Video , 2014, AVEC '14.

[31]  Tiranee Achalakul,et al.  Emotional healthcare system: Emotion detection by facial expressions using Japanese database , 2014, 2014 6th Computer Science and Electronic Engineering Conference (CEEC).