The challenges of joint attention

This article discusses the concept of joint attention and the different skills underlying its development. Research in developmental psychology clearly states that the development of skills to understand, manipulate and coordinate attentional behavior plays a pivotal role for imitation, social cognition and the development of language. However, beside the fact that joint attention has recently received an increasing interest in the robotics community, existing models concentrate only on partial and isolated elements of these phenomena. In the line of Tomasello’s research, we argue that joint attention is much more than simultaneous looking because it implies a shared intentional relation to the world. This requires skills for attention detection, attention manipulation, social coordination and, most importantly, intentional understanding. After defining joint attention and its challenges, the current state-of-the-art of robotic and computational models relevant for this issue is discussed in relation to a developmental timeline drawn from results in child studies. From this survey, we identify open issues and challenges that still need to be addressed to understand the development of the various aspects of joint attention and conclude with the potential contribution of robotic models.

[1]  F. H. Adler Cybernetics, or Control and Communication in the Animal and the Machine. , 1949 .

[2]  W. Ashby Design for a Brain , 1954 .

[3]  Marvin Minsky,et al.  A framework for representing knowledge , 1974 .

[4]  Marvin Minsky,et al.  A framework for representing knowledge" in the psychology of computer vision , 1975 .

[5]  P. L. Adams THE ORIGINS OF INTELLIGENCE IN CHILDREN , 1976 .

[6]  Roger C. Schank,et al.  Scripts, plans, goals and understanding: an inquiry into human knowledge structures , 1978 .

[7]  M. Cranach,et al.  Human ethology : claims and limits of a new discipline : contributions to the Colloquium , 1982 .

[8]  G. Butterworth,et al.  Towards a Mechanism of Joint Visual Attention in Human Infancy , 1980 .

[9]  S. Fiske,et al.  Social Psychology , 2019, Definitions.

[10]  K. Kaye,et al.  The Mental and Social Life of Babies , 1982 .

[11]  D. Stern The interpersonal world of infant , 1985 .

[12]  D. Dennett The Intentional Stance. , 1987 .

[13]  Werner Bohleber,et al.  The Interpersonal World of the Infant , 1989 .

[14]  D. Premack The infant's theory of self-propelled objects , 1990, Cognition.

[15]  G. Butterworth,et al.  What minds have in common is space : Spatial mechanisms serving joint visual attention in infancy , 1991 .

[16]  Jürgen Schmidhuber,et al.  Curious model-building control systems , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[17]  Dare A. Baldwin,et al.  Infants' ability to draw inferences about nonobvious object properties: evidence from exploratory play. , 1993, Child development.

[18]  H. Nothdurft The role of features in preattentive vision: Comparison of orientation, motion and color cues , 1993, Vision Research.

[19]  C. Moore,et al.  Social Understanding at the End of the First Year of Life , 1994 .

[20]  C. Moore,et al.  Joint attention : its origins and role in development , 1995 .

[21]  E. Keller,et al.  Refiguring Life: Metaphors of Twentieth-Century Biology , 1995 .

[22]  Michael A. Arbib,et al.  The handbook of brain theory and neural networks , 1995, A Bradford book.

[23]  A. Meltzoff Understanding the Intentions of Others: Re-Enactment of Intended Acts by 18-Month-Old Children. , 1995, Developmental psychology.

[24]  M. Tomasello Joint attention as social cognition. , 1995 .

[25]  Pawan Sinha,et al.  Perceiving and recognizing three-dimensional forms , 1996 .

[26]  Stefano Nolfi,et al.  Learning to Adapt to Changing Environments in Evolving Neural Networks , 1996, Adapt. Behav..

[27]  D. Povinelli,et al.  Mindblindness. An Essay on Autism and Theory of Mind Simon Baron-Cohen 1995 , 1996, Trends in Neurosciences.

[28]  B. Bertenthal,et al.  Origins and early development of perception, action, and representation. , 1996, Annual review of psychology.

[29]  A. Premack,et al.  Causal cognition : a multidisciplinary debate , 1996 .

[30]  Frank Dignum,et al.  Intentional Agents and Goal Formation , 1997, ATAL.

[31]  P. Rochat,et al.  Young infants' sensitivity to movement information specifying social causality , 1997 .

[32]  A. Gopnik,et al.  Early reasoning about desires: evidence from 14- and 18-month-olds. , 1997, Developmental psychology.

[33]  Jean-Arcady Meyer,et al.  Learning to Perceive the World as Articulated: An Approach for Hierarchical Learning in Sensory-Motor Systems , 1998 .

[34]  M. Tomasello,et al.  Fourteen-through 18-month-old infants di eren-tially imitate intentional and accidental actions , 1998 .

[35]  Sebastian Thrun,et al.  Exploration in active learning , 1998 .

[36]  Francesco Mondada,et al.  Evolutionary neurocontrollers for autonomous mobile robots , 1998, Neural Networks.

[37]  B. Scassellati Imitation and mechanisms of joint attention: a developmental structure for building social skills on a humanoid robot , 1999 .

[38]  D. Siegel The Developing Mind: Toward a Neurobiology of Interpersonal Experience. , 1999 .

[39]  Doina Precup,et al.  Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning , 1999, Artif. Intell..

[40]  Stefano Nolfi,et al.  Learning and Evolution , 1999, Auton. Robots.

[41]  Luc Steels,et al.  Collective Learning and Semiotic Dynamics , 1999, ECAL.

[42]  Stefano Nolfi,et al.  Learning to perceive the world as articulated: an approach for hierarchical learning in sensory-motor systems , 1998, Neural Networks.

[43]  乳児の joint attention と表情理解 , 1999 .

[44]  Kathleen Taylor,et al.  Attention, intention and salience in the posterior parietal cortex , 1999, Neurocomputing.

[45]  A. Woodward Infants' ability to distinguish between purposeful and non-purposeful behaviors , 1999 .

[46]  Stefan Schaal,et al.  Is imitation learning the route to humanoid robots? , 1999, Trends in Cognitive Sciences.

[47]  V. Gallese Action representaion and the inferior parietal lobule , 2000 .

[48]  Luc Steels,et al.  Aibo''s first words. the social learning of language and meaning. Evolution of Communication , 2002 .

[49]  J. Michael Herrmann,et al.  Learning predictive representations , 2000, Neurocomputing.

[50]  D. Kaufer,et al.  The Developing Mind: Toward a Neurobiology of Interpersonal Experience , 2000 .

[51]  L. Steels,et al.  Crucial factors in the origins of word-meaning , 2000 .

[52]  Kerstin Dautenhahn,et al.  Learning how to do things with imitation , 2000 .

[53]  Chrystopher L. Nehaniv Computation for Metaphors, Analogy, and Agents , 2000, Lecture Notes in Computer Science.

[54]  H. Kozima,et al.  A Robot that Learns to Communicate with Human Caregivers , 2001 .

[55]  Dario Floreano,et al.  Evolution of Plastic Control Networks , 2001, Auton. Robots.

[56]  J. Zlatev Meaning = Life (+ Culture): An outline of a unified biocultural theory of meaning , 2001 .

[57]  H. Wellman,et al.  Developing intentional understandings , 2001 .

[58]  Sorin Moga,et al.  Learning and communication via imitation: an autonomous robot perspective , 2001, IEEE Trans. Syst. Man Cybern. Part A.

[59]  James L. McClelland,et al.  Autonomous Mental Development by Robots and Animals , 2001, Science.

[60]  Matthew Quinn,et al.  Evolving Communication without Dedicated Communication Channels , 2001, ECAL.

[61]  G. Rizzolatti,et al.  Neurophysiological mechanisms underlying the understanding and imitation of action , 2001, Nature Reviews Neuroscience.

[62]  Jodie A. Baird,et al.  Making sense of human behavior: Action parsing and intentional inference , 2001 .

[63]  Jeffrey Mark Siskind,et al.  Grounding the Lexical Semantics of Verbs in Visual Perception using Force Dynamics and Event Logic , 1999, J. Artif. Intell. Res..

[64]  C. Breazeal,et al.  Robots that imitate humans , 2002, Trends in Cognitive Sciences.

[65]  Mitsuo Kawato,et al.  Multiple Model-Based Reinforcement Learning , 2002, Neural Computation.

[66]  Xiao Huang,et al.  Novelty and Reinforcement Learning in the Value System of Developmental Robots , 2002 .

[67]  Cynthia Breazeal,et al.  Designing sociable robots , 2002 .

[68]  M. Corbetta,et al.  Control of goal-directed and stimulus-driven attention in the brain , 2002, Nature Reviews Neuroscience.

[69]  M. Asada,et al.  A developmental approach accelerates learning of joint attention , 2002, Proceedings 2nd International Conference on Development and Learning. ICDL 2002.

[70]  Alex Pentland,et al.  Learning words from sights and sounds: a computational model , 2002, Cogn. Sci..

[71]  J. Nadel The Imitative Mind: Imitation and imitation recognition: Functional use in preverbal infants and nonverbal children with autism , 2002 .

[72]  Pierre-Yves Oudeyer,et al.  Robotic clicker training , 2002, Robotics Auton. Syst..

[73]  W. Prinz,et al.  The imitative mind : development, evolution, and brain bases , 2002 .

[74]  K. Dautenhahn,et al.  Imitation in Animals and Artifacts , 2002 .

[75]  Pierre-Yves Oudeyer,et al.  Motivational principles for visual know-how development , 2003 .

[76]  György Gergely,et al.  What should a robot learn from an infant? Mechanisms of action interpretation and observational learning in infancy , 2003, Connect. Sci..

[77]  L. Steels Evolving grounded communication for robots , 2003, Trends in Cognitive Sciences.

[78]  Hiroyuki Iizuka,et al.  Joint attention and dynam-ics repertoire in coupled dynamical recognizers , 2003 .

[79]  M. Tomasello,et al.  Chimpanzees understand psychological states – the question is which ones and to what extent , 2003, Trends in Cognitive Sciences.

[80]  Pierre-Yves Oudeyer,et al.  Maximizing Learning Progress: An Internal Reward System for Development , 2003, Embodied Artificial Intelligence.

[81]  Peter Ford Dominey Learning Grammatical Constructions from Narrated Video Events for Human-Robot Interaction , 2003 .

[82]  Alessandro Saffiotti,et al.  An introduction to the anchoring problem , 2003, Robotics Auton. Syst..

[83]  Stefano Nolfi,et al.  Evolving Mobile Robots Able to Display Collective Behaviors , 2003, Artificial Life.

[84]  Lincoln Smith,et al.  Evolving controllers for a homogeneous system of physical robots: structured cooperation with minimal sensors , 2003, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[85]  Minoru Asada,et al.  A constructive model for the development of joint attention , 2003, Connect. Sci..

[86]  Tetsuo Ono,et al.  Physical relation and expression: joint attention for human-robot interaction , 2003, IEEE Trans. Ind. Electron..

[87]  Tom Ziemke,et al.  Social Situatedness of Natural and Artificial Intelligence: Vygotsky and Beyond , 2003, Adapt. Behav..

[88]  Frédéric Kaplan,et al.  Who is Afraid of the Humanoid? Investigating Cultural Differences in the Acceptance of Robots , 2004, Int. J. Humanoid Robotics.

[89]  Jun Tani,et al.  On-line Imitative Interaction with a Humanoid Robot Using a Dynamic Neural Network Model of a Mirror System , 2004, Adapt. Behav..

[90]  Claire D’Este Sharing Meaning with Machines , 2004 .

[91]  Nuttapong Chentanez,et al.  Intrinsically Motivated Learning of Hierarchical Collections of Skills , 2004 .

[92]  Gordon Cheng,et al.  Discovering optimal imitation strategies , 2004, Robotics Auton. Syst..

[93]  David A. Cohn,et al.  Improving generalization with active learning , 1994, Machine Learning.

[94]  Douglas S. Blank,et al.  An Emergent Framework For Self-Motivation In Developmental Robotics , 2004 .

[95]  J. Triesch,et al.  A computational model of the emergence of gaze following , 2004 .

[96]  Pierre-Yves Oudeyer,et al.  The Playground Experiment: Task-Independent Development of a Curious Robot , 2005 .

[97]  Günther Palm,et al.  Biomimetic Neural Learning for Intelligent Robots - Intelligent Systems, Cognitive Robotics, and Neuroscience , 2005, Biomimetic Neural Learning for Intelligent Robots.

[98]  V. Hafner Interpersonal Maps and the Body Correspondence Problem , 2005 .

[99]  G. Lakoff,et al.  The Brain's concepts: the role of the Sensory-motor system in conceptual knowledge , 2005, Cognitive neuropsychology.

[100]  Frédéric Kaplan,et al.  Learning to Interpret Pointing Gestures: Experiments with Four-Legged Autonomous Robots , 2005, Biomimetic Neural Learning for Intelligent Robots.

[101]  M. Tomasello,et al.  Understanding and sharing intentions: The origins of cultural cognition , 2005, Behavioral and Brain Sciences.

[102]  Frédéric Kaplan,et al.  Les machines apprivoisées : comprendre les robots de loisir , 2005 .

[103]  S. Savage-Rumbaugh How monkeys see the world , 1992, International Journal of Primatology.

[104]  S. Harnad Symbol grounding problem , 1991, Scholarpedia.

[105]  Pierre-Yves Oudeyer,et al.  The progress drive hypothesis: an interpretation of early imitation , 2007 .