Eyes on the Mind: Investigating the Influence of Gaze Dynamics on the Perception of Others in Real-Time Social Interaction

Social gaze provides a window into the interests and intentions of others and allows us to actively point out our own. It enables us to engage in triadic interactions involving human actors and physical objects and to build an indispensable basis for coordinated action and collaborative efforts. The object-related aspect of gaze in combination with the fact that any motor act of looking encompasses both input and output of the minds involved makes this non-verbal cue system particularly interesting for research in embodied social cognition. Social gaze comprises several core components, such as gaze-following or gaze aversion. Gaze-following can result in situations of either “joint attention” or “shared attention.” The former describes situations in which the gaze-follower is aware of sharing a joint visual focus with the gazer. The latter refers to a situation in which gazer and gaze-follower focus on the same object and both are aware of their reciprocal awareness of this joint focus. Here, a novel interactive eye-tracking paradigm suited for studying triadic interactions was used to explore two aspects of social gaze. Experiments 1a and 1b assessed how the latency of another person’s gaze reactions (i.e., gaze-following or gaze version) affected participants’ sense of agency, which was measured by their experience of relatedness of these reactions. Results demonstrate that both timing and congruency of a gaze reaction as well as the other’s action options influence the sense of agency. Experiment 2 explored differences in gaze dynamics when participants were asked to establish either joint or shared attention. Findings indicate that establishing shared attention takes longer and requires a larger number of gaze shifts as compared to joint attention, which more closely seems to resemble simple visual detection. Taken together, novel insights into the sense of agency and the awareness of others in gaze-based interaction are provided.

[1]  Stephen V. Shepherd,et al.  Following Gaze: Gaze-Following Behavior as a Window into Social Cognition , 2010, Front. Integr. Neurosci..

[2]  M. Tarr,et al.  Virtual reality in behavioral neuroscience and beyond , 2002, Nature Neuroscience.

[3]  Jari K. Hietanen,et al.  Seeing direct and averted gaze activates the approach–avoidance motivational brain systems , 2008, Neuropsychologia.

[4]  Patrice D. Tremoulet,et al.  Perceptual causality and animacy , 2000, Trends in Cognitive Sciences.

[5]  M. Bender.  Perceptual interactions. , 1970, Modern trends in neurology.

[6]  M. Buehner,et al.  Causal Binding of Actions to Their Effects , 2009, Psychological science.

[7]  S. Baron-Cohen,et al.  The Empathy Quotient: An Investigation of Adults with Asperger Syndrome or High Functioning Autism, and Normal Sex Differences , 2004, Journal of autism and developmental disorders.

[8]  P. Haggard,et al.  Voluntary action and conscious awareness , 2002, Nature Neuroscience.

[9]  Nadim Joni Shah,et al.  Minds Made for Sharing: Initiating Joint Attention Recruits Reward-related Neurocircuitry , 2010, Journal of Cognitive Neuroscience.

[10]  L. Taylor,et al.  Human Agency in Social Cognitive Theory , 1989 .

[11]  Steven P. Tipper,et al.  Affective evaluations of objects are influenced by observed gaze direction and emotional expression , 2007, Cognition.

[12]  Matthew A. Paul,et al.  Gaze cuing and affective judgments of objects: I like what you look at , 2006, Psychonomic bulletin & review.

[13]  Jason A. Williams,et al.  Insincere Utterances and Gaze: Eye Contact during Sarcastic Statements , 2009, Perceptual and motor skills.

[14]  B. Hood,et al.  Tell-tale eyes: children's attribution of gaze aversion as a lying cue. , 2008, Developmental psychology.

[15]  M. Bellani,et al.  Virtual reality in autism: state of the art , 2011, Epidemiology and Psychiatric Sciences.

[16]  Simon Baron-Cohen,et al.  The systemizing quotient: an investigation of adults with Asperger syndrome or high-functioning autism, and normal sex differences. , 2003, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[17]  Elisabeth Pacherie,et al.  The Phenomenology of Joint Action: Self-Agency vs. Joint-Agency , 2012 .

[18]  F. Kaplan,et al.  The challenges of joint attention , 2006 .

[19]  M. Tomasello,et al.  Understanding and sharing intentions: The origins of cultural cognition , 2005, Behavioral and Brain Sciences.

[20]  B. Gold,et al.  Functional Dissociation in Frontal and Striatal Areas for Processing of Positive and Negative Reward Information , 2007, The Journal of Neuroscience.

[21]  Nicole C. Krämer,et al.  "It doesn't matter what you are!" Explaining social effects of agents and avatars , 2010, Comput. Hum. Behav..

[22]  Mark T. Waters,et al.  This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits distribution,andreproductioninanymedium,providedtheoriginalauthorandsourcearecredited.Thislicensedoesnot permit commercial exploitation or the creation of derivative works without sp , 2009 .

[23]  Simon B Eickhoff,et al.  Shall we do this together? Social gaze influences action control in a comparison group, but not in individuals with high-functioning autism , 2012, Autism : the international journal of research and practice.

[24]  N. Sarkar,et al.  Design of a Gaze-Sensitive Virtual Social Interactive System for Children With Autism , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[25]  K. Vogeley,et al.  Toward a second-person neuroscience 1 , 2013, Behavioral and Brain Sciences.

[26]  Christopher A. Dickinson,et al.  Coordinating spatial referencing using shared gaze , 2010, Psychonomic bulletin & review.

[27]  G. Bente,et al.  Avatar-Mediated Networking: Increasing Social Presence and Interpersonal Trust in Net-Based Collaborations , 2008 .

[28]  I. Biederman.  Perceiving Real-World Scenes , 1972, Science.

[29]  Daniel C. Richardson,et al.  The Dynamics of Reference and Shared Visual Attention , 2011, Front. Psychology.

[30]  P. Mundy,et al.  CURRENT DIRECTIONS IN PSYCHOLOGICAL SCIENCE Attention, Joint Attention, and Social Cognition , 2022 .

[31]  S. Tipper,et al.  Gaze cueing of attention: visual attention, social cognition, and individual differences. , 2007, Psychological bulletin.

[32]  M. Tomasello,et al.  Cooperation and human cognition: the Vygotskian intelligence hypothesis , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[33]  Hannah Hoch,et al.  Teaching children with autism to respond to and initiate bids for joint attention. , 2008, Journal of applied behavior analysis.

[34]  Stefanie I. Becker,et al.  Journal of Experimental Psychology : General “ Gaze Leading ” : Initiating Simulated Joint Attention Influences Eye Movements and Choice Behavior , 2012 .

[35]  N. Emery,et al.  The eyes have it: the neuroethology, function and evolution of social gaze , 2000, Neuroscience & Biobehavioral Reviews.

[36]  H. Aarts,et al.  When observing gaze shifts of others enhances object desirability. , 2010, Emotion.

[37]  M. Tomasello,et al.  Social cognition, joint attention, and communicative competence from 9 to 15 months of age. , 1998, Monographs of the Society for Research in Child Development.

[38]  Kai Vogeley,et al.  "Artificial humans": Psychology and neuroscience perspectives on embodiment and nonverbal communication , 2010, Neural Networks.

[39]  A. Newen,et al.  Beyond the comparator model: A multifactorial two-step account of agency , 2008, Consciousness and Cognition.

[40]  J. Nadel,et al.  Investigating social gaze as an action-perception online performance , 2012, Front. Hum. Neurosci..

[41]  C. Frith,et al.  The Neural Basis of Mentalizing , 2006, Neuron.

[42]  M. Argyle,et al.  Gaze and Mutual Gaze , 1994, British Journal of Psychiatry.

[43]  F. Vignemont,et al.  The sense of agency: A philosophical and empirical review of the “Who” system , 2004, Consciousness and Cognition.

[44]  Qing Yang,et al.  The latency of saccades, vergence, and combined eye movements in children and in adults. , 2002, Investigative ophthalmology & visual science.

[45]  Jean Carletta,et al.  Eyetracking for two-person tasks with manipulation of a virtual world , 2010, Behavior research methods.

[46]  U. Castiello,et al.  Toward You , 2010 .

[47]  M. Auvray,et al.  Perceptual interactions in a minimalist virtual environment , 2009 .

[48]  J. Loomis,et al.  Interpersonal Distance in Immersive Virtual Environments , 2003, Personality & social psychology bulletin.

[49]  Corey J. Bohil,et al.  Virtual reality in neuroscience research and therapy , 2011, Nature Reviews Neuroscience.

[50]  M. Crocker,et al.  Investigating joint attention mechanisms through spoken human–robot interaction , 2011, Cognition.

[51]  Kai Vogeley,et al.  A Non-Verbal Turing Test: Differentiating Mind from Machine in Gaze-Based Social Interaction , 2011, PloS one.

[52]  M. Mason,et al.  The Look of Love , 2005, Psychological science.

[53]  F. Michèl,et al.  Gaze discrimination is unimpaired in schizophrenia , 1998, Psychiatry Research.

[54]  Daniel Nettle,et al.  Agreeableness is related to social‐cognitive, but not social‐perceptual, theory of mind , 2008 .

[55]  Mark H. Johnson,et al.  Atypical eye contact in autism: Models, mechanisms and development , 2009, Neuroscience & Biobehavioral Reviews.

[56]  J. Loomis,et al.  Immersive virtual environment technology as a basic research tool in psychology , 1999, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[57]  D. Povinelli,et al.  Chimpanzees: Joint Visual Attention , 1996 .

[58]  J. Gibson,et al.  Perception of another person's looking behavior. , 1963, The American journal of psychology.

[59]  D. Stern,et al.  The goal and structure of mother-infant play. , 1974, Journal of the American Academy of Child Psychiatry.

[60]  Rebecca Saxe,et al.  Live face-to-face interaction during fMRI: A new tool for social cognitive neuroscience , 2010, NeuroImage.

[61]  G. Fink,et al.  It's in your eyes--using gaze-contingent stimuli to create truly interactive paradigms for social cognitive and affective neuroscience. , 2010, Social cognitive and affective neuroscience.

[62]  M. Saslow.  Effects of components of displacement-step stimuli upon latency for saccadic eye movement. , 1967, Journal of the Optical Society of America.