Facial expression recognition in ageing adults: from lab to ambient assisted living

This paper introduces an application that uses a webcam and aims to recognise emotions of ageing adults from their facial expression. Six basic emotions (Happiness, Sadness, Anger, Fear, Disgust and Surprise) as well as a Neutral state are distinguished. Active shape models are applied for feature extraction, the Cohn–Kanade, JAFFE and MMI databases are used for training, and support vector machines ($$\nu$$ν-SVM) are employed for facial expression classification. These six basic emotions are classified into three categories (Happiness, Negative-Emotion and Surprise), as Sadness, Anger, Fear and Disgust have been grouped into a single expression. The new three categories are found to be relevant for the purpose of detecting abnormalities in the ageing adult’s mood state. In the future, we will transfer the experience in a lab/research setup to a real ambient assisted living environment, which is the elderly person’s home. The application is devised to be the starting point to enhance the mood of the elderly people living alone at their homes by external stimuli.

[1]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[2]  Chen-Chiung Hsieh,et al.  A Facial Expression Classification System Based on Active Shape Model and Support Vector Machine , 2011, 2011 International Symposium on Computer Science and Society.

[3]  Xiangzhou Wang,et al.  Real-Time Facial Expression Recognition System Based-On Geometric Features , 2013 .

[4]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[5]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[6]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[7]  Mohammad Soleymani,et al.  A Multimodal Database for Affect Recognition and Implicit Tagging , 2012, IEEE Transactions on Affective Computing.

[8]  Chih-Jen Lin,et al.  Probability Estimates for Multi-class Classification by Pairwise Coupling , 2003, J. Mach. Learn. Res..

[9]  José Manuel Pastor,et al.  Smart environment architecture for emotion detection and regulation , 2016, J. Biomed. Informatics.

[10]  Yantao Tian,et al.  Facial expression recognition in video sequences , 2012, Proceedings of the 10th World Congress on Intelligent Control and Automation.

[11]  M. Bartlett,et al.  Machine Analysis of Facial Expressions , 2007 .

[12]  Bir Bhanu,et al.  A biologically inspired approach for fusing facial expression and appearance for emotion recognition , 2012, 2012 19th IEEE International Conference on Image Processing.

[13]  Antonio Fernández-Caballero,et al.  Engineering the development of systems for multisensory monitoring and activity interpretation , 2014, Int. J. Syst. Sci..

[14]  José Manuel Pastor,et al.  Improvement of the Elderly Quality of Life and Care through Smart Emotion Regulation , 2014, IWAAL.

[15]  Xiaodong Gu,et al.  Happy-Sad Expression Recognition Using Emotion Geometry Feature and Support Vector Machine , 2009, ICONIP.

[16]  Antonio Fernández-Caballero,et al.  Facial Expression Recognition from Webcam Based on Active Shape Models and Support Vector Machines , 2014, IWAAL.

[17]  Antonio Fernández-Caballero,et al.  A Framework for Recognizing and Regulating Emotions in the Elderly , 2014, IWAAL.

[18]  Jiliu Zhou,et al.  Geometric feature based facial expression recognition using multiclass support vector machines , 2009, 2009 IEEE International Conference on Granular Computing.

[19]  Deepak Ghimire,et al.  Geometric Feature-Based Facial Expression Recognition in Image Sequences Using Multi-Class AdaBoost and Support Vector Machines , 2013, Sensors.

[20]  Zhen Li,et al.  Emotion recognition from an ensemble of features , 2011, Face and Gesture 2011.

[21]  Mikkel B. Stegmann,et al.  Analysis and segmentation of face images using point annotations and linear subspace techniques, in , 2002 .

[22]  José Manuel Pastor,et al.  Software Architecture for Smart Emotion Recognition and Regulation of the Ageing Adult , 2016, Cognitive Computation.

[23]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[24]  Antonio Fernández-Caballero,et al.  Thermal-Infrared Pedestrian ROI Extraction through Thermal and Motion Information Fusion , 2014, Sensors.

[25]  M. H. Purnomo,et al.  Facial emotional expressions recognition based on Active Shape Model and Radial Basis Function Network , 2012, 2012 IEEE International Conference on Computational Intelligence for Measurement Systems and Applications (CIMSA) Proceedings.

[26]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[27]  Lin Mei,et al.  A face recognition algorithm based on ASM and Gabor features of key points , 2013, International Conference on Graphic and Image Processing.

[28]  Vinod Chandran,et al.  Discovering the Best Feature Extraction and Selection Algorithms for Spontaneous Facial Expression Recognition , 2012, 2012 IEEE International Conference on Multimedia and Expo.

[29]  Rogério Schmidt Feris,et al.  Manifold Based Analysis of Facial Expression , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[30]  Om Mishra,et al.  Face Recognition Using RBF Kernel Based Support Vector Machine , 2012 .

[31]  N. Alugupally,et al.  Analysis of landmarks in recognition of face expressions , 2011, Pattern Recognition and Image Analysis.

[32]  Ilias Maglogiannis,et al.  Face detection and recognition of natural human emotion using Markov random fields , 2007, Personal and Ubiquitous Computing.

[33]  B. Prabhakaran,et al.  Real-Time Mobile Facial Expression Recognition System -- A Case Study , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.