Facial expressions based error detection for smart environment using deep learning

Creating innovative applications of ambient intelligence for smart environments became a real challenge. Indeed, the existing systems are considered as being obtrusive. Therefore, the ongoing trends consist in finding new methods which are less intrusive while being accurate and computer efficient. To that end, we introduce an ingenious way to assist the elderly and people with cognitive impairment in their daily life using emotions through facial expressions. Our hypothesis assumes the correlation between expressed emotions and the resulting errors in daily activities. In order to verify it, we introduce an experimentation protocol with neurotypical subjects. In this paper, we mainly focus on designing an approach for facial expression recognition using Convolutional Neural Network. The carried experiments with benchmark datasets provided promising results. In fact, the proposed approach outperforms the existing methods in terms of accuracy. Moreover, our approach has been implemented in an embedded system with limited resources. The main objective of this paper is to provide an effective approach recognizing facial expressions that will be used for our further experimentations.

[1]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[2]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[3]  Martin J. Russell,et al.  CogWatch: Automatic prompting system for stroke survivors during activities of daily living , 2016, J. Innov. Digit. Ecosyst..

[4]  Dong-Soo Kwon,et al.  Baseline CNN structure analysis for facial expression recognition , 2016, 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[5]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[6]  Zhiyong Feng,et al.  Facial expression recognition via deep learning , 2014, 2014 International Conference on Smart Computing.

[7]  Abdenour Bouzouane,et al.  A New Approach of Facial Expression Recognition for Ambient Assisted Living , 2016, PETRA.

[8]  Abdenour Bouzouane,et al.  A KEYHOLE PLAN RECOGNITION MODEL FOR ALZHEIMER'S PATIENTS: FIRST RESULTS , 2007, Appl. Artif. Intell..

[9]  A. Mihailidis,et al.  The COACH prompting system to assist older adults with dementia through handwashing: An efficacy study , 2008, BMC geriatrics.

[10]  J. Broekens,et al.  Assistive social robots in elderly care: a review , 2009 .

[11]  Sven Wachsmuth,et al.  TEBRA: An Automatic Prompting System for Persons with Cognitive Disabilities in Brushing Teeth , 2013, HEALTHINF.

[12]  Wei-Yang Lin,et al.  Facial expression recognition using bag of distances , 2013, Multimedia Tools and Applications.

[13]  Qingshan Liu,et al.  Learning active facial patches for expression analysis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Wei Li,et al.  A deep-learning approach to facial expression recognition with candid images , 2015, 2015 14th IAPR International Conference on Machine Vision Applications (MVA).

[15]  P. Ekman Are there basic emotions? , 1992, Psychological review.

[16]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[17]  D. Lundqvist,et al.  Karolinska Directed Emotional Faces , 2015 .

[18]  Stefano Ferilli,et al.  Towards an Empathic Social Robot for Ambient Assisted Living , 2015, ESSEM@AAMAS.

[19]  Josephine Sullivan,et al.  One millisecond face alignment with an ensemble of regression trees , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Yongzhao Zhan,et al.  Multi-pose facial expression recognition based on SURF boosting , 2015, 2015 International Conference on Affective Computing and Intelligent Interaction (ACII).

[21]  Aliaa A. A. Youssif,et al.  Automatic Facial Expression Recognition System Based on Geometric and Appearance Features , 2011, Comput. Inf. Sci..

[22]  Radhika M. Pai,et al.  Automatic Facial Expression Recognition Using DCNN , 2016 .

[23]  C. Baum,et al.  Cognitive performance in senile dementia of the Alzheimer's type: the Kitchen Task Assessment. , 1993, The American journal of occupational therapy : official publication of the American Occupational Therapy Association.