A psychometric analysis of the reading the mind in the eyes test: toward a brief form for research and applied settings

The Reading the Mind in the Eyes Test is a popular measure of individual differences in Theory of Mind that is often applied in the assessment of particular clinical populations (primarily, individuals on the autism spectrum). However, little is known about the test's psychometric properties, including factor structure, internal consistency, and convergent validity evidence. We present a psychometric analysis of the test followed by an evaluation of other empirically proposed and statistically identified structures. We identified, and cross-validated in a second sample, an adequate short-form solution that is homogeneous with adequate internal consistency, and is moderately related to Cognitive Empathy, Emotion Perception, and strongly related to Vocabulary. We recommend the use of this short-form solution in normal adults as a more precise measure over the original version. Future revisions of the test should seek to reduce the test's reliance on one's vocabulary and evaluate the short-form structure in clinical populations.

[1]  R. N. Spreng,et al.  The Toronto Empathy Questionnaire: Scale Development and Initial Validation of a Factor-Analytic Solution to Multiple Empathy Measures , 2009, Journal of personality assessment.

[2]  S. Nowicki,et al.  Individual differences in the nonverbal communication of affect: The diagnostic analysis of nonverbal accuracy scale , 1994 .

[3]  P. Bentler.  Comparative Fit Indices in Structural Models , 1990 .

[4]  Zvi Drezner,et al.  Model Specification Searches Using Ant Colony Optimization Algorithms , 2003 .

[5]  K. Harkness,et al.  Mental state decoding in past major depression: Effect of sad versus happy mood induction , 2010 .

[6]  L. Stephen Miller,et al.  Executive Function Mechanisms of Theory of Mind , 2011, Journal of autism and developmental disorders.

[7]  S. Baron-Cohen,et al.  Test-retest reliability of the ‘Reading the Mind in the Eyes’ test: a one-year follow-up study , 2013, Molecular Autism.

[8]  Simon Baron-Cohen,et al.  The “Reading the Mind in the Eyes” test: Systematic review of psychometric properties and a validation study in Italy , 2013, Cognitive neuropsychiatry.

[9]  M. Sigman,et al.  The cortical processing of facial emotional expression is associated with social cognition skills and executive functioning: A preliminary study , 2011, Neuroscience Letters.

[10]  N. Emery,et al.  The eyes have it: the neuroethology, function and evolution of social gaze , 2000, Neuroscience & Biobehavioral Reviews.

[11]  Rex B. Kline,et al.  Principles and Practice of Structural Equation Modeling , 1998 .

[12]  David A. Wood,et al.  What a performance , 2004 .

[13]  L. Tucker,et al.  A reliability coefficient for maximum likelihood factor analysis , 1973 .

[14]  R. P. McDonald,et al.  Test Theory: A Unified Treatment , 1999 .

[15]  Self-Concept Variables.  Sex Differences in , 2016 .

[16]  S. Baron-Cohen,et al.  The latent structure of cognitive and emotional empathy in individuals with autism, first-degree relatives and typical individuals , 2014, Molecular Autism.

[17]  A. Miu,et al.  Emotional face processing in neurotypicals with autistic traits: Implications for the broad autism phenotype , 2012, Psychiatry Research.

[18]  O. Wilhelm,et al.  An emotion-differentiated perspective on empathy with the emotion specific empathy questionnaire , 2014, Front. Psychol..

[19]  V. Savalei.  The Relationship Between Root Mean Square Error of Approximation and Model Misspecification in Confirmatory Factor Analysis Models , 2012 .

[20]  M. Hyland,et al.  Inter-Item Correlation Frequency Distribution Analysis: A Method for Evaluating Scale Dimensionality , 1993 .

[21]  M. Voracek,et al.  Lack of correlation between digit ratio (2D:4D) and Baron-Cohen's "Reading the Mind in the Eyes" test, empathy, systemising, and autism-spectrum quotients in a general population sample. , 2006 .

[22]  D. Premack,et al.  Does the chimpanzee have a theory of mind? , 1978, Behavioral and Brain Sciences.

[23]  Edward E. Rigdon,et al.  CFI versus RMSEA: A comparison of two fit indexes for structural equation modeling , 1996 .

[24]  B. Repacholi,et al.  Introduction Individual Differences in Theory of Mind: What Are We Investigating? , 2003 .

[25]  C. A. Baker,et al.  Meta-Analysis Reveals Adult Female Superiority in "Reading the Mind in the Eyes Test" , 2013 .

[26]  J. M. Cortina,et al.  What Is Coefficient Alpha? An Examination of Theory and Applications , 1993 .

[27]  T. Little,et al.  To Parcel or Not to Parcel: Exploring the Question, Weighing the Merits , 2002 .

[28]  M. Tomasello,et al.  Does the chimpanzee have a theory of mind? 30 years later , 2008, Trends in Cognitive Sciences.

[29]  S. Baron-Cohen,et al.  Does the autistic child have a “theory of mind” ? , 1985, Cognition.

[30]  Adam J. Berinsky,et al.  Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk , 2012, Political Analysis.

[31]  T. Little.  Mean and Covariance Structures (MACS) Analyses of Cross-Cultural Data: Practical and Theoretical Issues. , 1997, Multivariate behavioral research.

[32]  D. Watson,et al.  Constructing validity: Basic issues in objective scale development , 1995 .

[33]  Rebecca C. Knickmeyer,et al.  Sex Differences in the Brain: Implications for Explaining Autism , 2005, Science.

[34]  Ruth B. Ekstrom,et al.  Manual for kit of factor-referenced cognitive tests , 1976 .

[35]  J. Astington.  Sometimes necessary, never sufficient: False-belief understanding and social competence. , 2003 .

[36]  Moritz Heene,et al.  Masking misfit in confirmatory factor analysis by increasing unique variances: a cautionary note on the usefulness of cutoff values of fit indices. , 2011, Psychological methods.

[37]  S. Baron-Cohen,et al.  The “Reading the Mind in Films” Task: Complex emotion recognition in adults with and without autism spectrum conditions , 2006, Social neuroscience.

[38]  C. Gillberg,et al.  The Reading the Mind in the Eyes Test: Test–retest reliability of a Swedish version , 2009, Cognitive neuropsychiatry.

[39]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[40]  K. Harkness,et al.  Enhanced accuracy of mental state decoding in dysphoric college students , 2005 .

[41]  André Beauducel,et al.  On the Performance of Maximum Likelihood Versus Means and Variance Adjusted Weighted Least Squares Estimation in CFA , 2006 .

[42]  A. Senju,et al.  ‘Autistic’ Traits in Non-Autistic Japanese Populations: Relationships with Personality Traits and Cognitive Ability , 2006, Journal of autism and developmental disorders.

[43]  Ian B. Hickie,et al.  Intranasal Oxytocin Improves Emotion Recognition for Youth with Autism Spectrum Disorders , 2010, Biological Psychiatry.

[44]  P. Bentler,et al.  Comparative fit indexes in structural models. , 1990, Psychological bulletin.

[45]  J. Astington,et al.  A longitudinal study of the relation between language and theory-of-mind development. , 1999, Developmental psychology.

[46]  O. I. Lovaas,et al.  The autistic child , 1977 .

[47]  Reginald B. Adams,et al.  Cross-cultural Reading the Mind in the Eyes: An fMRI Investigation , 2010, Journal of Cognitive Neuroscience.

[48]  B. Desgranges,et al.  Age effects on different components of theory of mind , 2011, Consciousness and Cognition.

[49]  O. Almkvist,et al.  Psychometric data on the Eyes Test, the Faux Pas Test, and the Dewey Social Stories Test in a population-based Swedish adult sample , 2012 .

[50]  M. Posner,et al.  Orienting of Attention* , 1980, The Quarterly journal of experimental psychology.

[51]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[52]  Walter L Leite,et al.  Item Selection for the Development of Short Forms of Scales Using an Ant Colony Optimization Algorithm , 2008, Multivariate behavioral research.

[53]  Reading the Mind in the Eyes Test (Children's Version): A Comparison Study between Children with Typical Development, Children with High-Functioning Autism and Typically Developed Adults , 2014, Folia Phoniatrica et Logopaedica.

[54]  M. W. Richardson,et al.  The theory of the estimation of test reliability , 1937 .

[55]  S. Baron-Cohen,et al.  Measuring empathy: reliability and validity of the Empathy Quotient , 2004, Psychological Medicine.

[56]  J. Henry,et al.  Evidence for deficits in facial affect recognition and theory of mind in multiple sclerosis , 2009, Journal of the International Neuropsychological Society.

[57]  Alex Martin,et al.  Facial Emotion Recognition in Autism Spectrum Disorders: A Review of Behavioral and Neuroimaging Studies , 2010, Neuropsychology Review.

[58]  E. Bora,et al.  Theory of mind impairment in schizophrenia: Meta-analysis , 2009, Schizophrenia Research.

[59]  I. Gold,et al.  The Reading the Mind in the Eyes test: validation of a French version and exploration of cultural variations in a multi-ethnic city , 2014, Cognitive neuropsychiatry.

[60]  D. Flora,et al.  An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. , 2004, Psychological methods.

[61]  R. Foley,et al.  A Maternal Influence on Reading the Mind in the Eyes Mediated by Executive Function: Differential Parental Influences on Full and Half-Siblings , 2011, PloS one.

[62]  Oliver Wilhelm,et al.  Methods matter: Testing competing models for designing short-scale Big-Five assessments , 2015 .

[63]  Facundo Manes,et al.  A neuropsychological battery to detect specific executive and social cognitive impairments in early frontotemporal dementia. , 2009, Brain : a journal of neurology.

[64]  S. Dehning,et al.  Comparative cross-sectional study of empathy among first year and final year medical students in Jimma University, Ethiopia: Steady state of the heart and opening of the eyes , 2012, BMC medical education.

[65]  E. Peterson,et al.  The Eyes Test as a Measure of Individual Differences: How much of the Variance Reflects Verbal IQ? , 2012, Front. Psychology.

[66]  R. Zachariae,et al.  PTSD and Impaired Eye Expression Recognition: A Preliminary Study , 2009 .

[67]  A. Young,et al.  Reading the mind from eye gaze , 2002, Neuropsychologia.

[68]  Krista Casler,et al.  Separate but equal? A comparison of participants and data gathered via Amazon's MTurk, social media, and face-to-face behavioral testing , 2013, Comput. Hum. Behav..

[69]  Mark H. Davis,et al.  A Multidimensional Approach to Individual Differences in Empathy , 1980 .

[70]  T. Keenan.  Individual Differences in Theory of Mind: The Preschool Years and Beyond , 2004 .

[71]  S. Baron-Cohen,et al.  The “Reading the Mind in the Eyes” Test: Investigation of Psychometric Properties and Test–Retest Reliability of the Persian Version , 2015, Journal of Autism and Developmental Disorders.

[72]  B. Bushman,et al.  The Relationship Between Narcissistic Exploitativeness, Dispositional Empathy, and Emotion Recognition Abilities , 2014 .

[73]  S. Baron-Cohen,et al.  Another advanced test of theory of mind: evidence from very high functioning adults with autism or asperger syndrome. , 1997, Journal of child psychology and psychiatry, and allied disciplines.

[74]  H. Wellman,et al.  Meta-analysis of theory-of-mind development: the truth about false belief. , 2001, Child development.

[75]  Alev Girli.  Psychometric Properties of the Turkish Child and Adult Form of "Reading the Mind in the Eyes Test" , 2014 .

[76]  E. Ates,et al.  Investigation of the reliability of the "reading the mind in the eyes test" in a Turkish population. , 2011, Turk psikiyatri dergisi = Turkish journal of psychiatry.

[77]  J. Astington,et al.  Cognitive Factors and Family Structure Associated with Theory of Mind Development in Young Children. , 1996 .

[78]  William Revelle,et al.  Estimating Generalizability to a Latent Variable Common to All of a Scale's Indicators: A Comparison of Estimators for ωh , 2006 .

[79]  L. Angeles.  Evaluating Cutoff Criteria of Model Fit Indices for Latent Variable Models with Binary and Continuous Outcomes , 2002 .

[80]  S. Baron-Cohen,et al.  Is there an innate gaze module? Evidence from human neonates , 2000 .

[81]  D. Massaro,et al.  Development and Evaluation of a Computer-Animated Tutor for Vocabulary and Language Learning in Children with Autism , 2003, Journal of autism and developmental disorders.

[82]  P. Philippot,et al.  The “Reading the Mind in the Eyes” test as a new way to explore complex emotions decoding in alcohol dependence , 2011, Psychiatry Research.

[83]  Elizabeth J. Austin,et al.  Associations of trait and ability emotional intelligence with performance on Theory of Mind tasks in an adult sample , 2010 .

[84]  S. Baron-Cohen,et al.  The "Reading the Mind in the Eyes" Test revised version: a study with normal adults, and adults with Asperger syndrome or high-functioning autism. , 2001, Journal of child psychology and psychiatry, and allied disciplines.

[85]  B. Repacholi,et al.  Individual differences in theory of mind : implications for typical and atypical development , 2003 .

[86]  J. H. Steiger.  Statistically based tests for the number of common factors , 1980 .