Differentiating Duchenne from non-Duchenne smiles using active appearance models

Face-related biometrics research in recent years has moved from attempting merely to recognize faces, and even doing so under varying conditions, to considering a wide variety of aspects such as dynamics, gesture, aging, and expression. The state of an individual's face is a revealing indicator that may be used for soft biometrics, active authentication, deception detection, response feedback, and other areas of interface. One related psychological indicator is the Duchenne smile that usually indicates a genuine, spontaneous, or enjoyed emotional state rather than a forced or posed state, as likely expressed by a non-Duchenne smile. Differentiating between these is a useful task to automate for a variety of reasons. This paper discusses a classification technique that achieves higher recognition rates than previously published for similar comparisons.

[1]  Gwen Littlewort,et al.  Faces of pain: automated measurement of spontaneousallfacial expressions of genuine and posed pain , 2007, ICMI '07.

[2]  J. Cohn,et al.  All Smiles are Not Created Equal: Morphology and Timing of Smiles Perceived as Amused, Polite, and Embarrassed/Nervous , 2009, Journal of nonverbal behavior.

[3]  Hatice Gunes,et al.  How to distinguish posed from spontaneous smiles using geometric features , 2007, ICMI '07.

[4]  Bernhard Schölkopf,et al.  A tutorial on v-support vector machines , 2005 .

[5]  Tsuhan Chen,et al.  The painful face - Pain expression recognition using active appearance models , 2009, Image Vis. Comput..

[6]  P. Ekman.  Darwin, Deception, and Facial Expression , 2003, Annals of the New York Academy of Sciences.

[7]  T. Bayes.  An essay towards solving a problem in the doctrine of chances , 2003 .

[8]  Gwen Littlewort,et al.  An approach to automatic recognition of spontaneous facial actions , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[9]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[10]  Eva Krumhuber,et al.  Moving Smiles: The Role of Dynamic Components for the Perception of the Genuineness of Smiles , 2005 .

[11]  Gwen Littlewort,et al.  Automated measurement of children's facial expressions during problem solving tasks , 2011, Face and Gesture 2011.

[12]  Ralph Gross,et al.  Active appearance models with occlusion , 2006, Image Vis. Comput..

[13]  J. Movellan,et al.  Are Your Eyes Smiling? Detecting genuine smiles with support vector machines and Gabor wavelets , 2001 .

[14]  Simon Lucey,et al.  Face alignment through subspace constrained mean-shifts , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[15]  Marian Stewart Bartlett,et al.  Measuring the Perceived Difficulty of a Lecture Using Automatic Facial Expression Recognition , 2008, Intelligent Tutoring Systems.

[16]  P. Ekman,et al.  The Duchenne smile: emotional expression and brain physiology. II. , 1990, Journal of personality and social psychology.

[17]  John McDonald,et al.  Estimation of the Temporal Dynamics of Posed and Spontaneous Facial Expression Formation Using LLE , 2009, 2009 13th International Machine Vision and Image Processing Conference.

[18]  Karen L. Schmidt,et al.  Demonstration of facial communication of emotion through telehospice videophone contact. , 2011, Telemedicine journal and e-health : the official journal of the American Telemedicine Association.

[19]  Javier R. Movellan,et al.  Automated facial affect analysis for one-on-one tutoring applications , 2011, Face and Gesture 2011.

[20]  Nicu Sebe,et al.  Authentic facial expression analysis , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[21]  Marian Stewart Bartlett,et al.  Automated Facial Expression Measurement: recent Applications to Basic Research in Human Behavior, Learning, and Education , 2011 .

[22]  Arman Savran,et al.  Bosphorus Database for 3D Face Analysis , 2008, BIOID.

[23]  Paul L. Rosin,et al.  Assessing the Uniqueness and Permanence of Facial Actions for Use in Biometric Applications , 2010, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[24]  Thomas J. Watson,et al.  An empirical study of the naive Bayes classifier , 2001 .

[25]  Tom Janicki,et al.  Active Appearance Models for Affect Recognition using Facial Expressions , 2010 .

[26]  M. Pantic,et al.  Induced Disgust , Happiness and Surprise : an Addition to the MMI Facial Expression Database , 2010 .

[27]  Irina Rish,et al.  An empirical study of the naive Bayes classifier , 2001 .

[28]  P. Perona,et al.  Automatic Extraction of Facial Action Codes , 2001 .

[29]  P. Ekman,et al.  Facial expression and the affective component of cynical hostility in male coronary heart disease patients. , 1998, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[30]  Timothy F. Cootes,et al.  Active Appearance Models , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  J. Cohn,et al.  Automated face analysis by feature point tracking has high concurrent validity with manual FACS coding. , 1999, Psychophysiology.

[32]  Adam E. Gaweda,et al.  Individual identification based on facial dynamics during expressions using active-appearance-based Hidden Markov Models , 2011, Face and Gesture 2011.

[33]  Gwen Littlewort,et al.  Discrimination of Moderate and Acute Drowsiness Based on Spontaneous Facial Expressions , 2010, 2010 20th International Conference on Pattern Recognition.

[34]  Arman Savran,et al.  Estimation of facial action intensities on 2D and 3D data , 2011, 2011 19th European Signal Processing Conference.

[35]  Gwen Littlewort,et al.  Insights on Spontaneous Facial Expressions from Automatic Expression Measurement , 2010 .

[36]  Ivan Bratko,et al.  Microarray data mining with visual programming , 2005, Bioinform..

[37]  Timothy F. Cootes,et al.  Model-based methods in analysis of Biomedical images. , 1999 .

[38]  Fernando De la Torre,et al.  Facial Expression Analysis , 2011, Visual Analysis of Humans.

[39]  Jason Weston,et al.  A user's guide to support vector machines. , 2010, Methods in molecular biology.

[40]  David Cristinacce,et al.  Automatic feature localisation with constrained local models , 2008, Pattern Recognit..

[41]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[42]  Chih-Jen Lin,et al.  A Practical Guide to Support Vector Classication , 2008 .

[43]  P. Ekman,et al.  Smiles when lying. , 1988, Journal of personality and social psychology.

[44]  Qiang Ji,et al.  A Unified Probabilistic Framework for Spontaneous Facial Action Modeling and Understanding , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  J. Hietanen,et al.  Facial and emotional reactions to Duchenne and non-Duchenne smiles. , 1998, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[46]  P. Ekman.  Emotions Revealed: Recognizing Faces and Feelings to Improve Communication and Emotional Life , 2003 .

[47]  Hichem Sahli,et al.  Automatic recognition of lower facial action units , 2010, MB '10.

[48]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[49]  Gwen Littlewort,et al.  Machine learning methods for fully automatic recognition of facial expressions and facial actions , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[50]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[51]  Jonathon Shlens,et al.  A Tutorial on Principal Component Analysis , 2014, ArXiv.

[52]  J. Movellan,et al.  SmileMaze: A Tutoring System in Real-Time Facial Expression Perception and Production in Children with Autism Spectrum Disorder , 2008 .

[53]  A. Manstead,et al.  Can Duchenne smiles be feigned? New evidence on felt and false smiles. , 2009, Emotion.

[54]  Gwen Littlewort,et al.  Recognizing facial expression: machine learning and application to spontaneous behavior , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[55]  Takeo Kanade,et al.  Recognizing Action Units for Facial Expression Analysis , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[56]  Harry Zhang,et al.  The Optimality of Naive Bayes , 2004, FLAIRS.

[57]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[58]  Simon Lucey,et al.  Investigating Spontaneous Facial Action Recognition through AAM Representations of the Face , 2007 .

[59]  Jeffrey F. Cohn,et al.  The Timing of Facial Motion in posed and Spontaneous Smiles , 2003, Int. J. Wavelets Multiresolution Inf. Process..

[60]  Bernhard Schölkopf,et al.  New Support Vector Algorithms , 2000, Neural Computation.

[61]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[62]  Gwen Littlewort,et al.  The computer expression recognition toolbox (CERT) , 2011, Face and Gesture 2011.

[63]  P. Ekman.  Telling lies: clues to deceit in the marketplace , 1985 .

[64]  Adrian Hilton,et al.  Visual Analysis of Humans - Looking at People , 2013 .

[65]  Takeo Kanade,et al.  Facial Expression Analysis , 2011, AMFG.

[66]  P. Ekman,et al.  Felt, false, and miserable smiles , 1982 .

[67]  P. Ekman,et al.  Behavioral Markers and Recognizability of the Smile of Enjoyment , 1993 .

[68]  N. Fox,et al.  Patterns of brain electrical activity during facial signs of emotion in 10-month-old infants. , 1988 .

[69]  Andrew P. Bradley,et al.  The use of the area under the ROC curve in the evaluation of machine learning algorithms , 1997, Pattern Recognit..

[70]  Chih-Jen Lin,et al.  A tutorial on?-support vector machines , 2005 .

[71]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[72]  Gwen Littlewort,et al.  Fully Automatic Facial Action Recognition in Spontaneous Behavior , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[73]  Qiang Ji,et al.  Modeling and exploiting the spatio-temporal facial action dependencies for robust spontaneous facial expression recognition , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[74]  Karl Pearson F.R.S..  LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[75]  Maja Pantic,et al.  Web-based database for facial expression analysis , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[76]  Javier R. Movellan,et al.  Towards Automatic Recognition of Spontaneous Facial Actions , 2003 .

[77]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[78]  Gwen Littlewort,et al.  Automatic coding of facial expressions displayed during posed and genuine pain , 2009, Image Vis. Comput..

[79]  Simon Baker,et al.  Active Appearance Models Revisited , 2004, International Journal of Computer Vision.

[80]  Maja Pantic,et al.  Spontaneous vs. posed facial behavior: automatic analysis of brow actions , 2006, ICMI '06.

[81]  P. Ekman,et al.  Voluntary Smiling Changes Regional Brain Activity , 1993 .