CBR Tagging of Emotions from Facial Expressions

Mobility and context-awareness are two active research directions that open new potential to recommender systems. Usage of dynamically enriched information from the user context leads the system to find better solutions that are adapted to the specific situations. In this paper we focus on the difficult problem of dynamically acquiring the emotional context about the user during a recommendation process. We use the fact that emotions are tightly connected with facial expressions and it is difficult for people to hide emotions in facial expressions. We describe PhotoMood, a CBR system that uses gestures to identify emotions in faces, and present preliminary experiments with MadridLive, a mobile and context aware recommender system for leisure activities in Madrid. In the experiments, the momentary emotion of a user is dynamically detected from pictures of the facial expression taken unobtrusively with the front facing camera of the mobile device.

[1]  Rana El Kaliouby,et al.  Self-Cam: feedback from what would be your social partner , 2006, SIGGRAPH '06.

[2]  Tomaso A. Poggio,et al.  Example-Based Learning for View-Based Human Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Jeffrey F. Cohn,et al.  Foundations of human computing: facial expression and emotion , 2006, ICMI '06.

[4]  Osvaldo Cairó,et al.  MICAI 2000: Advances in Artificial Intelligence , 2000, Lecture Notes in Computer Science.

[5]  Susan Craw,et al.  Genetic Algorithms to Optimise CBR Retrieval , 2000, EWCBR.

[6]  Ilias Maglogiannis,et al.  Face detection and recognition of natural human emotion using Markov random fields , 2007, Personal and Ubiquitous Computing.

[7]  P. M. Chavan,et al.  Real Time Emotion Recognition through Facial Expressions for Desktop Devices , 2013 .

[8]  E. Diener,et al.  Experience Sampling: Promises and Pitfalls, Strengths and Weaknesses , 2003 .

[9]  Thomas S. Huang,et al.  Human face detection in a complex background , 1994, Pattern Recognit..

[10]  P. Peer,et al.  Human skin color clustering for face detection , 2003, The IEEE Region 8 EUROCON 2003. Computer as a Tool..

[11]  Maja Pantic,et al.  Facial action recognition for facial expression analysis from static face images , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[12]  Guodong Guo,et al.  Support vector machines for face recognition , 2001, Image Vis. Comput..

[13]  Stephen J. Anderson,et al.  Investigating the structure of autobiographical memories , 1993 .

[14]  Anisha Halder,et al.  Facial expression and EEG signal based classification of emotion , 2014, International Conference on Electronics, Communication and Instrumentation (ICECI).

[15]  Poulcheria Benou,et al.  Context-Aware Query Processing in Ad-Hoc Environments of Peers , 2008, J. Electron. Commer. Organ..

[16]  Maja Pantic,et al.  Automatic Analysis of Facial Expressions: The State of the Art , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Monson H. Hayes,et al.  Hidden Markov models for face recognition , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[18]  Franc Solina,et al.  COLOR-BASED FACE DETECTION IN THE "15 SECONDS OF FAME" ART INSTALLATION , 2003 .

[19]  Saïda Bouakaz,et al.  Extraction of Facial Feature Points Using Cumulative Histogram , 2012, ArXiv.

[20]  Mike McGrath,et al.  ACM SIGGRAPH 2006 Research posters , 2006, SIGGRAPH 2006.

[21]  J. G. Taylor,et al.  Emotion recognition in human-computer interaction , 2005, Neural Networks.

[22]  Takeo Kanade,et al.  Neural Network-Based Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  A. Ortony,et al.  What's basic about basic emotions? , 1990, Psychological review.

[24]  Jih Pin Yeh,et al.  Face Detection Based on Skin Color Segmentation and SVM Classification , 2008, 2008 Second International Conference on Secure System Integration and Reliability Improvement.

[25]  Kenji Suzuki,et al.  Analysis of Social Smile Sharing Using a Wearable Device that Captures Distal Electromyographic Signals , 2012, 2012 Third International Conference on Emerging Security Technologies.

[26]  Evangelos Karapanos,et al.  Modeling Users' Experiences with Interactive Systems , 2012, Studies in Computational Intelligence.

[27]  Abderrahim Elmoataz,et al.  Image and Signal Processing, 4th International Conference, ICISP 2010, Trois-Rivières, QC, Canada, June 30-July 2, 2010. Proceedings , 2010, ICISP.

[28]  Oleg Seredin,et al.  Comparative Testing of Face Detection Algorithms , 2010, ICISP.

[29]  George N. Votsis,et al.  Emotion recognition in human-computer interaction , 2001, IEEE Signal Process. Mag..

[30]  Bruce A. Draper,et al.  Recognizing faces with PCA and ICA , 2003, Comput. Vis. Image Underst..

[31]  Qingshan Liu,et al.  Face recognition using kernel based fisher discriminant analysis , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[32]  Nicu Sebe,et al.  Facial expression recognition from video sequences: temporal and static modeling , 2003, Comput. Vis. Image Underst..

[33]  Alex Pentland,et al.  Artifical Intelligence for Human Computing, ICMI 2006 and IJCAI 2007 International Workshops, Banff, Canada, November 3, 2006, Hyderabad, India, January 6, 2007, Revised Seleced and Invited Papers , 2007, Artifical Intelligence for Human Computing.

[34]  Guillermo Jiménez-Díaz,et al.  Social factors in group recommender systems , 2013, TIST.

[35]  Roberto Marcondes Cesar Junior,et al.  Detection and Tracking of Facial Features in Video Sequences , 2000, MICAI.

[36]  Francesco Ricci,et al.  Location-aware music recommendation , 2013, International Journal of Multimedia Information Retrieval.

[37]  Tieniu Tan,et al.  A new face detection method based on shape information , 2000, Pattern Recognit. Lett..

[38]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[39]  Francesco Ricci,et al.  Context-Aware Recommender Systems , 2011, AI Mag..

[40]  Dorothy S. Strickland,et al.  Computer as Tool , 1987 .

[41]  J. Russell.  Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies. , 1994, Psychological bulletin.

[42]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[43]  F. Solina.  15 seconds of fame , 2004, Leonardo.

[44]  Kuo-Chin Fan,et al.  Triangle-based approach to the detection of human face , 2001, Pattern Recognit..