Characterizing facial expressions by grammars of action unit sequences - A first investigation using ABL

We investigate the application of grammar inference to the analysis of facial expressions to discover underlying sequential regularities characteristic for a specific mental state. The input consists of sequences of action units (AUs), which represent basic facial signals. The typical classification task for facial expression analysis is to assign a set of AUs its corresponding mental state, e.g., an emotion. To our knowledge, there is no research investigating whether there is diagnostic information in the sequence in which the AUs occur in a given time interval. Our study is based on data of facial expressions of pain obtained in a psychological experiment with 347 pain episodes of 86 subjects represented as sequences of AUs. We applied the Alignment-Based Learning (ABL) approach to infer the underlying grammar for the set of all AUs which occurred in the sequences and for a reduced alphabet of the relevant AUs only. We used 10-fold cross-validation to estimate performance and we extended ABL with a frequency-based heuristics to reduce the number of grammar rules by eliminating such rules which do not contribute significantly to performance. The resulting grammar for the reduced AU alphabet provides a first approximation for a "grammar of pain".

[1]  Tsuhan Chen,et al.  The painful face - Pain expression recognition using active appearance models , 2009, Image Vis. Comput..

[2]  Thomas Hadjistavropoulos,et al.  A psychophysical investigation of the facial action coding system as an index of pain variability among older adults with and without Alzheimer's disease. , 2007, Pain medicine.

[3]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[4]  Eugene Charniak,et al.  Statistical language learning , 1997 .

[5]  K. Prkachin.  The consistency of facial expressions of pain: a comparison across modalities , 1992, Pain.

[6]  Ute Schmid,et al.  Inductive rule learning on the knowledge level , 2011, Cognitive Systems Research.

[7]  Georgios Paliouras,et al.  Guest editorial to the special issue on grammatical inference , 2006, Machine Learning.

[8]  M. Kunz,et al.  The facial expression of pain in patients with dementia , 2007, Pain.

[9]  Menno van Zaanen.  Implementing Alignment-Based Learning , 2002, ICGI.

[10]  Zahir M. Hussain,et al.  Automatic facial expression recognition: feature extraction and selection , 2010, Signal, Image and Video Processing.

[11]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[12]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[13]  Beatrice Santorini,et al.  Building a Large Annotated Corpus of English: The Penn Treebank , 1993, CL.

[14]  Sridha Sridharan,et al.  Automatically Detecting Pain in Video Through Facial Action Units , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[15]  Douglas B. Lenat,et al.  The Role of Heuristics in Learning by Discovery: Three Case Studies , 1983 .

[16]  Colin de la Higuera,et al.  A bibliographical study of grammatical inference , 2005, Pattern Recognit..

[17]  Jeroen Geertzen,et al.  Problems with Evaluation of Unsupervised Empirical Grammatical Inference Systems , 2008, ICGI.

[18]  Petra Perner,et al.  Data Mining - Concepts and Techniques , 2002, Künstliche Intell..

[19]  Takeo Kanade,et al.  Automated facial expression recognition based on FACS action units , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[20]  Kostas Karpouzis,et al.  Affective Intelligence: The Human Face of AI , 2009, Artificial Intelligence: An International Perspective.

[21]  Ronen Feldman,et al.  The Data Mining and Knowledge Discovery Handbook , 2005 .

[22]  Michael Siebers,et al.  Applying Grammar Inference To Identify Generalized Patterns of Facial Expressions of Pain , 2012, ICGI.

[23]  Beat Fasel,et al.  Automati Fa ial Expression Analysis: A Survey , 1999 .

[24]  James F. Allen.  Maintaining knowledge about temporal intervals , 1983, CACM.

[25]  Rajesh Parekh,et al.  Automata Induction, Grammar Inference, and Language Acquisition , 2000 .

[26]  John Shawe-Taylor,et al.  Introduction to the Special Topic on Grammar Induction, Representation of Language and Language Learning , 2011, J. Mach. Learn. Res..

[27]  Alaa A. Kharbouch,et al.  Three models for the description of language , 1956, IRE Trans. Inf. Theory.

[28]  Michael Siebers,et al.  On the Relevance of Sequence Information for Decoding Facial Expressions of Pain and Disgust An Avatar Study , 2013 .

[29]  Dominique Estival,et al.  Theoretical and Practical Experiences with Alignment-Based Learning , 2003 .

[30]  Stephen Muggleton,et al.  Learning from Positive Data , 1996, Inductive Logic Programming Workshop.

[31]  Menno van Zaanen,et al.  Bootstrapping structure into language : alignment-based learning , 2001, ArXiv.

[32]  Tim Oates,et al.  INTRODUCTION: SPECIAL ISSUE ON APPLICATIONS OF GRAMMATICAL INFERENCE , 2008, Appl. Artif. Intell..

[33]  Jeffrey F. Cohn.  Advances in Behavioral Science Using Automated Facial Image Analysis and Synthesis [Social Sciences] , 2010, IEEE Signal Processing Magazine.