Understanding eye movements in face recognition using hidden Markov models.

We use a hidden Markov model (HMM) based approach to analyze eye movement data in face recognition. HMMs are statistical models that are specialized in handling time-series data. We conducted a face recognition task with Asian participants, and model each participant's eye movement pattern with an HMM, which summarized the participant's scan paths in face recognition with both regions of interest and the transition probabilities among them. By clustering these HMMs, we showed that participants' eye movements could be categorized into holistic or analytic patterns, demonstrating significant individual differences even within the same culture. Participants with the analytic pattern had longer response times, but did not differ significantly in recognition accuracy from those with the holistic pattern. We also found that correct and wrong recognitions were associated with distinctive eye movement patterns; the difference between the two patterns lies in the transitions rather than locations of the fixations alone.

[1]  Joseph H. Goldberg,et al.  Scanpath clustering and aggregation , 2010, ETRA.

[2]  Nicola C. Anderson,et al.  Curious eyes: Individual differences in personality predict eye movement behavior in scene-viewing , 2012, Cognition.

[3]  G. Cottrell,et al.  Two Fixations Suffice in Face Recognition , 2008, Psychological science.

[4]  A. Young,et al.  Matching Familiar and Unfamiliar Faces on Internal and External Features , 1985, Perception.

[5]  R. C. Oldfield.  The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[6]  Rachael E. Jack,et al.  Culture Shapes How We Look at Faces , 2008, PloS one.

[7]  L. Stark,et al.  Scanpaths in saccadic eye movements while viewing and recognizing patterns. , 1971, Vision research.

[8]  O. Grüsser,et al.  Gaze motor asymmetries in the perception of faces during a memory task , 1993, Neuropsychologia.

[9]  Rachael E. Jack,et al.  Social Experience Does Not Abolish Cultural Diversity in Eye Movements , 2011, Front. Psychology.

[10]  J. Henderson.  Eye movement control during visual object processing: effects of initial fixation position and semantic constraint. , 1993, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[11]  P. Schyns,et al.  Local Jekyll and Global Hyde : The Dual Identity of Face Identification , 2011 .

[12]  John M Henderson,et al.  Stable individual differences across images in human saccadic eye movements. , 2008, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[13]  H. Ellis,et al.  Identification of Familiar and Unfamiliar Faces from Internal and External Features: Some Implications for Theories of Face Recognition , 1979, Perception.

[14]  D. I. Perrett,et al.  Are the perceptual biases found in chimeric face processing reflected in eye-movement patterns? , 2005, Neuropsychologia.

[15]  Carrick C. Williams,et al.  Eye movements are functional during face learning , 2005, Memory & cognition.

[16]  Christian Maier,et al.  Objective measures of IS usage behavior under conditions of experience and pressure using eye fixation data , 2013, ICIS.

[17]  Michael E. Holmes,et al.  Attention to repeated images on the World-Wide Web: Another look at scanpath theory , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[18]  Antoni B. Chan,et al.  Clustering hidden Markov models with variational HEM , 2012, J. Mach. Learn. Res..

[19]  K. Rayner.  Eye Guidance in Reading: Fixation Locations within Words , 1979, Perception.

[20]  Antoni B. Chan,et al.  The variational hierarchical EM algorithm for clustering hidden Markov models , 2012, NIPS.

[21]  D. S. Wooding,et al.  Fixation Patterns Made during Brief Examination of Two-Dimensional Images , 1997, Perception.

[22]  A. Gale,et al.  Eye Movement Strategies Involved in Face Perception , 1977, Perception.

[23]  S R Ellis,et al.  Statistical Dependency in Visual Scanning , 1986, Human factors.

[24]  B. Laeng,et al.  Eye scanpaths during visual imagery reenact those of perception of the same visual scene , 2002 .

[25]  Werner Van Belle,et al.  Anchoring gaze when categorizing faces’ sex: Evidence from eye-tracking data , 2009, Vision Research.

[26]  J. Piven,et al.  Visual Scanning of Faces in Autism , 2002, Journal of autism and developmental disorders.

[27]  D. M. Titterington,et al.  VARIATIONAL BAYESIAN ANALYSIS FOR HIDDEN MARKOV MODELS , 2009 .

[28]  Sébastien Miellet,et al.  iMap: a novel method for statistical fixation mapping of eye movement data , 2011, Behavior research methods.

[29]  David Barber,et al.  Bayesian reasoning and machine learning , 2012 .

[30]  Miwa Hayashi.  Hidden Markov Models to identify pilot instrument scanning and attention patterns , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[31]  Michel Wedel,et al.  Global and local covert visual attention: Evidence from a bayesian hidden markov model , 2003 .

[32]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[33]  Matthew F. Peterson,et al.  Individual Differences in Eye Movements During Face Identification Reflect Observer-Specific Optimal Points of Fixation , 2013, Psychological science.

[34]  Ilpo Kojo,et al.  Using hidden Markov model to uncover processing states from eye movements in information search tasks , 2008, Cognitive Systems Research.

[35]  R. Pieters,et al.  Visual Attention to Repeated Print Advertising: A Test of Scanpath Theory , 1999 .

[36]  Frédéric Gosselin,et al.  Bubbles: a technique to reveal the use of information in recognition tasks , 2001, Vision Research.

[37]  Natalie M. Myres,et al.  Distinctive Paleo-Indian Migration Routes from Beringia Marked by Two Rare mtDNA Haplogroups , 2009, Current Biology.

[38]  Jason J S Barton,et al.  Information Processing during Face Recognition: The Effects of Familiarity, Inversion, and Morphing on Scanning Fixations , 2006, Perception.

[39]  S M Luria,et al.  Comparison of Eye Movements over Faces in Photographic Positives and Negatives , 1978, Perception.

[40]  Rachael E. Jack,et al.  Cultural Confusions Show that Facial Expressions Are Not Universal , 2009, Current Biology.