Sty in the Mind’s Eye: A Meta-Analytic Investigation of the Nomological Network and Internal Consistency of the “Reading the Mind in the Eyes” Test

The Reading the Mind in the Eyes Test (RMET) is the most popular adult measure of individual differences in theory of mind. We present a meta-analytic investigation of the test’s psychometric properties (k = 119 effect sizes, 61 studies, ntotal = 8,611 persons). Using random effects models, we found the internal consistency of the test was acceptable (α = .73). However, the RMET was more strongly related with emotion perception (r = .33, ρ = .48) relative to alternative theory of mind measures (r = .29, ρ = .39), and weakly to moderately related with vocabulary (r = .25, ρ = .32), cognitive empathy (r = .14, ρ = .20), and affective empathy (r = .13, ρ = .19). Overall, we conclude that the RMET operates rather as emotion perception measure than as theory of mind measure, challenging the interpretation of RMET results.

[1]  J. Pustejovsky,et al.  Evaluating meta-analytic methods to detect selective reporting in the presence of dependent effect sizes. , 2020, Psychological methods.

[2]  K. Harkness,et al.  Valence in the Reading the Mind in the Eyes Task. , 2020, Psychological assessment.

[3]  Y. Rossetti,et al.  What Do Theory-of-Mind Tasks Actually Measure? Theory and Practice , 2020, Perspectives on psychological science : a journal of the Association for Psychological Science.

[4]  O. Wilhelm,et al.  Overarching Principles for the Organization of Socioemotional Constructs , 2020 .

[5]  G. J. Lewis,et al.  Emotion recognition ability: Evidence for a supramodal factor and its links to social cognition , 2020, Cognition.

[6]  S. Olderbak,et al.  Four-Branch Model of Ability Emotional Intelligence With Fluid and Crystallized Intelligence: A Meta-Analysis of Relations , 2018, Emotion Review.

[7]  Shir Tragash,et al.  reading the mind in the eyes , 2018 .

[8]  L. F. Barrett,et al.  Universality Reconsidered: Diversity in Making Meaning of Facial Expressions , 2018, Current directions in psychological science.

[9]  F.-Xavier Alario,et al.  Consensus Analysis for Populations With Latent Subgroups: Applying Multicultural Consensus Theory and Model-Based Clustering With CCTpack , 2018 .

[10]  Laurie R Santos,et al.  Psychopaths fail to automatically take the perspective of others , 2018, Proceedings of the National Academy of Sciences.

[11]  Suzanne Jak,et al.  Accounting for Missing Correlation Coefficients in Fixed-Effects MASEM , 2018, Multivariate behavioral research.

[12]  M. Baciu,et al.  Role of Two Types of Syntactic Embedding in Belief Attribution in Adults with or without Asperger Syndrome , 2017, Front. Psychol..

[13]  O. Wilhelm,et al.  Emotion Perception and Empathy: An Individual Differences Test of Relations , 2017, Emotion.

[14]  K. Scherer,et al.  Introducing a short version of the Geneva Emotion Recognition Test (GERT-S): Psychometric properties and construct validation , 2016, Behavior research methods.

[15]  O. Wilhelm,et al.  Individual Differences in Performance on Comprehension and Knowledge Tests With and Without Passages and Questions , 2016 .

[16]  P. Salovey,et al.  The Ability Model of Emotional Intelligence: Principles and Updates , 2016 .

[17]  G. Bird,et al.  Theory of Mind Is Not Theory of Emotion: A Cautionary Note on the Reading the Mind in the Eyes Test , 2016, Journal of abnormal psychology.

[18]  Galit Yovel,et al.  A Revised Neural Framework for Face Processing. , 2015, Annual review of vision science.

[19]  Richard D. Roberts,et al.  A psychometric analysis of the reading the mind in the eyes test: toward a brief form for research and applied settings , 2015, Front. Psychol..

[20]  Michael V. Lombardo,et al.  The “Reading the Mind in the Eyes” Test: Complete Absence of Typical Sex Difference in ~400 Men and Women with Autism , 2015, PloS one.

[21]  W. Sommer,et al.  Perceiving and remembering emotional facial expressions — A basic facet of emotional intelligence , 2015 .

[22]  Donald W. Pfaff,et al.  Deconstructing and reconstructing theory of mind , 2015, Trends in Cognitive Sciences.

[23]  Tricia Z. King,et al.  The role of emotion perception in adaptive functioning of people with autism spectrum disorders , 2015, Autism : the international journal of research and practice.

[24]  G. Csukly,et al.  Investigating the state-like and trait-like characters of social cognition in schizophrenia: A short term follow-up study , 2014, Schizophrenia Research.

[25]  Alev Girli Psychometric Properties of the Turkish Child and Adult Form of "Reading the Mind in the Eyes Test" , 2014 .

[26]  S. Baron-Cohen,et al.  The latent structure of cognitive and emotional empathy in individuals with autism, first-degree relatives and typical individuals , 2014, Molecular Autism.

[27]  E. Chambers,et al.  The development of an emotion lexicon for the coffee drinking experience , 2014 .

[28]  O. Wilhelm,et al.  An emotion-differentiated perspective on empathy with the emotion specific empathy questionnaire , 2014, Front. Psychol..

[29]  A. Fabio,et al.  Comparing ability and self-report trait emotional intelligence, fluid intelligence, and personality traits in career decision , 2014 .

[30]  K. Scherer,et al.  Introducing the Geneva emotion recognition test: an example of Rasch-based test development. , 2014, Psychological assessment.

[31]  Andrea Hildebrandt,et al.  Test battery for measuring the perception and recognition of facial expressions of emotion , 2014, Front. Psychol..

[32]  C. A. Baker,et al.  Eyes and IQ: A meta-analysis of the relationship between intelligence and “Reading the Mind in the Eyes” , 2014 .

[33]  John O. Willis,et al.  Wechsler Abbreviated Scale of Intelligence , 2014 .

[34]  John O. Willis,et al.  Wechsler Adult Intelligence Scale–Fourth Edition , 2014 .

[35]  Simon Baron-Cohen,et al.  The “Reading the Mind in the Eyes” test: Systematic review of psychometric properties and a validation study in Italy , 2013, Cognitive neuropsychiatry.

[36]  K. Barchard,et al.  When proportion consensus scoring works , 2013 .

[37]  C. A. Baker,et al.  Meta-Analysis Reveals Adult Female Superiority in "Reading the Mind in the Eyes Test" , 2013 .

[38]  P. Jackson,et al.  On what ground do we mentalize? Characteristics of current tasks and sources of information that contribute to mentalizing judgments. , 2013, Psychological assessment.

[39]  G. Juckel,et al.  Mental state decoding and mental state reasoning in recently detoxified alcohol-dependent individuals , 2013, Psychiatry Research.

[40]  G. Horstmann,et al.  Coherence between Emotion and Facial Expression: Evidence from Laboratory Experiments , 2013 .

[41]  J. Fernández-Dols,et al.  Emotion and Expression: Naturalistic Studies , 2013 .

[42]  S. Dehning,et al.  Comparative cross-sectional study of empathy among first year and final year medical students in Jimma University, Ethiopia: Steady state of the heart and opening of the eyes , 2012, BMC medical education.

[43]  I. Apperly What is “theory of mind”? Concepts, cognitive processes and individual differences , 2012, Quarterly journal of experimental psychology.

[44]  O. Almkvist,et al.  Psychometric data on the Eyes Test, the Faux Pas Test, and the Dewey Social Stories Test in a population-based Swedish adult sample , 2012 .

[45]  P. Philippot,et al.  The “Reading the Mind in the Eyes” test as a new way to explore complex emotions decoding in alcohol dependence , 2011, Psychiatry Research.

[46]  A. Young,et al.  Understanding person perception. , 2011, British journal of psychology.

[47]  B. Mesquita,et al.  Context in Emotion Perception , 2011 .

[48]  E. Walker,et al.  Diagnostic and Statistical Manual of Mental Disorders , 2013 .

[49]  Noel A. Card Applied Meta-Analysis for Social Science Research , 2011 .

[50]  L. Stephen Miller,et al.  Executive Function Mechanisms of Theory of Mind , 2011, Journal of autism and developmental disorders.

[51]  J. Braithwaite,et al.  Seeing it their way: evidence for rapid and involuntary computation of what other people see. , 2010, Journal of experimental psychology. Human perception and performance.

[52]  E. Viding,et al.  Etiology of different developmental trajectories of callous-unemotional traits. , 2010, Journal of the American Academy of Child and Adolescent Psychiatry.

[53]  S. Channon,et al.  Mentalising and social problem-solving after brain injury , 2010, Neuropsychological rehabilitation.

[54]  Elizabeth Tipton,et al.  Robust variance estimation in meta‐regression with dependent effect size estimates , 2010, Research synthesis methods.

[55]  L. Hedges,et al.  The Handbook of Research Synthesis and Meta-Analysis , 2009 .

[56]  K. Scherer,et al.  Emotion recognition from expressions in face, voice, and body: the Multimodal Emotion Recognition Test (MERT). , 2009, Emotion.

[57]  M. Hardy-Bayle,et al.  Quantitative assessment of attribution of intentions to others in schizophrenia using an ecological video-based task: A comparison with manic and depressed patients , 2009, Psychiatry Research.

[58]  E. Bora,et al.  Theory of mind impairment in schizophrenia: Meta-analysis , 2009, Schizophrenia Research.

[59]  C. Gillberg,et al.  The Reading the Mind in the Eyes Test: Test–retest reliability of a Swedish version , 2009, Cognitive neuropsychiatry.

[60]  John O. Willis,et al.  Wechsler Adult Intelligence Scale–Third Edition , 2008 .

[61]  K. Scherer,et al.  The World of Emotions is not Two-Dimensional , 2007, Psychological science.

[62]  James R Bettman,et al.  The effects of nonconsciously priming emotion concepts on behavior. , 2007, Journal of personality and social psychology.

[63]  K. Scherer,et al.  Interaction effects of perceived gaze direction and dynamic facial expression: Evidence for appraisal theories of emotion , 2007 .

[64]  S. Herpertz,et al.  Oxytocin Improves “Mind-Reading” in Humans , 2007, Biological Psychiatry.

[65]  J. Russell,et al.  Bias in consensus scoring, with examples from ability emotional intelligence tests. , 2006, Psicothema.

[66]  M. Voracek,et al.  Lack of correlation between digit ratio (2D:4D) and Baron-Cohen's "Reading the Mind in the Eyes" test, empathy, systemising, and autism-spectrum quotients in a general population sample. , 2006 .

[67]  Jan K. Woike,et al.  Introducing MASC: A Movie for the Assessment of Social Cognition , 2006, Journal of autism and developmental disorders.

[68]  Jonathan A C Sterne,et al.  Regression Methods to Detect Publication and Other Bias in Meta-Analysis , 2006 .

[69]  S. Baron-Cohen,et al.  The Cambridge Mindreading (CAM) Face-Voice Battery: Testing Complex Emotion Recognition in Adults with and without Asperger Syndrome , 2006, Journal of autism and developmental disorders.

[70]  R. Blair Responding to the emotions of others: Dissociating forms of empathy through the study of typical and psychiatric populations , 2005, Consciousness and Cognition.

[71]  K. Harkness,et al.  Enhanced accuracy of mental state decoding in dysphoric college students , 2005 .

[72]  Mike W-L Cheung,et al.  Meta-analytic structural equation modeling: a two-stage approach. , 2005, Psychological methods.

[73]  S. Baron-Cohen,et al.  Measuring empathy: reliability and validity of the Empathy Quotient , 2004, Psychological Medicine.

[74]  R. Kleck,et al.  Perceived Gaze Direction and the Processing of Facial Displays of Emotion , 2003, Psychological science.

[75]  R. Gur,et al.  Facial emotion recognition in schizophrenia: intensity effects and error pattern. , 2003, The American journal of psychiatry.

[76]  M. Brüne Social Cognition and Behaviour in Schizophrenia , 2003 .

[77]  J. J. Ryan,et al.  Scoring Reliability on the Wechsler Adult Intelligence Scale-Third Edition (WAIS-III) , 2003, Assessment.

[78]  Skye McDonald,et al.  TASIT: A New Clinical Tool for Assessing Social Perception After Traumatic Brain Injury , 2003, The Journal of head trauma rehabilitation.

[79]  Robin L. Nabi The theoretical versus the lay meaning of disgust: Implications for emotion research , 2002 .

[80]  Hillary Anger Elfenbein,et al.  On the universality and cultural specificity of emotion recognition: a meta-analysis. , 2002, Psychological bulletin.

[81]  H. Wellman,et al.  Meta-analysis of theory-of-mind development: the truth about false belief. , 2001, Child development.

[82]  S. Baron-Cohen,et al.  The "Reading the Mind in the Eyes" Test revised version: a study with normal adults, and adults with Asperger syndrome or high-functioning autism. , 2001, Journal of child psychology and psychiatry, and allied disciplines.

[83]  Sarah Crawford,et al.  The effects of anterior lesions on performance on a story comprehension test: left anterior impairment on a theory of mind-type task , 2000, Neuropsychologia.

[84]  Richard J. Davidson,et al.  Exploring Hindu Indian Emotion Expressions: Evidence for Accurate Recognition by Americans and Indians , 2000, Psychological science.

[85]  Bruce Thompson,et al.  Psychometrics is Datametrics: the Test is not Reliable , 2000 .

[86]  R. Tweedie,et al.  A Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis , 2000 .

[87]  Max Coltheart,et al.  Mentalising, schizotypy, and schizophrenia , 1999, Cognition.

[88]  J. Russell,et al.  Core affect, prototypical emotional episodes, and other things called emotion: dissecting the elephant. , 1999, Journal of personality and social psychology.

[89]  H. Brownell,et al.  Acquired `theory of mind' impairments following stroke , 1999, Cognition.

[90]  R. Knight,et al.  Frontal Lobe Contributions to Theory of Mind , 1998, Journal of Cognitive Neuroscience.

[91]  Robin I. M. Dunbar,et al.  Theory-of-mind deficits and causal attributions , 1998 .

[92]  M. Bell,et al.  Positive and negative affect recognition in schizophrenia: a comparison with substance abuse and normal control subjects , 1997, Psychiatry Research.

[93]  S. Baron-Cohen,et al.  Another advanced test of theory of mind: evidence from very high functioning adults with autism or asperger syndrome. , 1997, Journal of child psychology and psychiatry, and allied disciplines.

[94]  S. Baron-Cohen,et al.  Is There a "Language of the Eyes"? Evidence from Normal Adults, and Adults with Autism or Asperger Syndrome , 1997 .

[95]  Y. Sarfati,et al.  Attribution of intentions to others in people with schizophrenia: a non-verbal exploration with comic strips , 1997, Schizophrenia Research.

[96]  C. Frith,et al.  Exploring ‘theory of mind’ in people with schizophrenia , 1996, Psychological Medicine.

[97]  C. Frith,et al.  Schizophrenia, symptomatology and social inference: Investigating “theory of mind” in people with schizophrenia , 1995, Schizophrenia Research.

[98]  F. Happé An advanced test of theory of mind: Understanding of story characters' thoughts and feelings by able autistic, mentally handicapped, and normal children and adults , 1994, Journal of autism and developmental disorders.

[99]  S. Nowicki,et al.  Individual differences in the nonverbal communication of affect: The diagnostic analysis of nonverbal accuracy scale , 1994 .

[100]  R. Lane,et al.  The Levels of Emotional Awareness Scale: a cognitive-developmental measure of emotion. , 1990, Journal of personality assessment.

[101]  M. S. Patel,et al.  An introduction to meta-analysis. , 1989, Health Policy.

[102]  R. Orwin A fail-safe N for effect size in meta-analysis. , 1983 .

[103]  D. Premack,et al.  Does the chimpanzee have a theory of mind? , 1978, Behavioral and Brain Sciences.

[104]  M. Snyder Self-monitoring of expressive behavior. , 1974 .

[105]  A. Mehrabian,et al.  A measure of emotional empathy. , 1972, Journal of personality.

[106]  M. Amelang,et al.  Grundlagen diagnostischer Verfahren , 2021, Psychologische Diagnostik.

[107]  A. Hill,et al.  Wechsler Test of Adult Reading , 2017 .

[108]  L. Faust,et al.  Understanding Other Minds Perspectives From Developmental Cognitive Neuroscience , 2016 .

[109]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[110]  S. McDonald,et al.  Cognitive factors underpinning poor expressive communication skills after traumatic brain injury: theory of mind or executive function? , 2014, Neuropsychology.

[111]  Andrew H Kemp,et al.  The Impact of a Single Administration of Intranasal Oxytocin on the Recognition of Basic Emotions in Humans: A Meta-Analysis , 2013, Neuropsychopharmacology.

[112]  Reginald B. Adams,et al.  Cross-cultural Reading the Mind in the Eyes: An fMRI Investigation , 2010, Journal of Cognitive Neuroscience.

[113]  C. Batson These things called empathy: Eight related but distinct phenomena. , 2009 .

[114]  E. Meer,et al.  Emotion Studies in HCI – a New Approach , 2005 .

[115]  D. Borsboom,et al.  The concept of validity. , 2004, Psychological review.

[116]  Jeanette Altarriba,et al.  The distinctiveness of emotion concepts: a comparison between emotion, abstract, and concrete words. , 2004, The American journal of psychology.

[117]  William Ickes,et al.  Measuring empathic accuracy. , 2001 .

[118]  Mark H. Davis,et al.  A Multidimensional Approach to Individual Differences in Empathy , 1980 .

[119]  P. Ekman Pictures of Facial Affect , 1976 .

[120]  Ruth B. Ekstrom,et al.  Manual for kit of factor-referenced cognitive tests , 1976 .

[121]  P. Ekman,et al.  Unmasking the face : a guide to recognizing emotions from facial clues , 1975 .

[122]  J. L. Harrison,et al.  The Government Printing Office , 1968, American Journal of Pharmaceutical Education.

[123]  G. A. Ferguson,et al.  HUMAN ABILITIES. , 1965, Annual review of psychology.

[124]  A. Sias Interpretation of educational measurements. , 1928 .

[125]  F. G.,et al.  An Introduction to the Theory of Mental and Social Measurements , 1904, Nature.