Investigating multimodal real-time patterns of joint attention in an hri word learning task

Joint attention - the idea that humans make inferences from observable behaviors of other humans by attending to the objects and events that these others humans attend to - has been recognized as a critical component in human-robot interactions. While various HRI studies showed that having robots to behave in ways that support human recognition of joint attention leads to better behavioral outcomes on the human side, there are no studies that investigate the detailed time course of interactive joint attention processes. In this paper, we present the results from an HRI study that investigates the exact time course of human multi-modal attentional processes during an HRI word learning task in an unprecedented way. Using novel data analysis techniques, we are able to demonstrate that the temporal details of human attentional behavior are critical for understanding human expectations of joint attention in HRI and that failing to do so can force humans into assuming unnatural behaviors.

[1]  Andrea Lockerd Thomaz,et al.  Teaching and working with robots as a collaboration , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[2]  Matthew W. Crocker,et al.  Visual attention in spoken human-robot interaction , 2009, 2009 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[3]  Bilge Mutlu,et al.  A Storytelling Robot: Modeling and Evaluation of Human-like Gaze Behavior , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[4]  Z. Griffin.  Why Look? Reasons for Eye Movements Related to Language Production. , 2004 .

[5]  Linda B. Smith,et al.  Active Information Selection: Visual Attention Through the Hands , 2009, IEEE Transactions on Autonomous Mental Development.

[6]  B. Scassellati.  Imitation and mechanisms of joint attention: a developmental structure for building social skills on a humanoid robot , 1999 .

[7]  Takayuki Kanda,et al.  Providing route directions: Design of robot's utterance, gesture, and timing , 2009, 2009 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[8]  Tetsuo Ono,et al.  Physical relation and expression: joint attention for human-robot interaction , 2001, Proceedings 10th IEEE International Workshop on Robot and Human Interactive Communication. ROMAN 2001 (Cat. No.01TH8591).

[9]  Matthias Scheutz,et al.  Incremental natural language processing for HRI , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[10]  Matthias Scheutz,et al.  First steps toward natural human-like HRI , 2007, Auton. Robots.

[11]  Paul D. Allopenna,et al.  Tracking the Time Course of Spoken Word Recognition Using Eye Movements: Evidence for Continuous Mapping Models , 1998 .

[12]  C. Moore,et al.  Joint attention : its origins and role in development , 1995 .

[13]  Takayuki Kanda,et al.  Footing in human-robot conversations: How robots might shape participant roles using gaze cues , 2009, 2009 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[14]  Charles R. Crowell,et al.  Robot social presence and gender: Do females view robots differently than males? , 2008, 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[15]  Matthias Scheutz,et al.  The utility of affect expression in natural language interactions in joint human-robot tasks , 2006, HRI '06.

[16]  Magdalena D. Bugajska,et al.  An Embodied Model of Infant Gaze-Following , 2009 .

[17]  M. Tomasello,et al.  Social cognition, joint attention, and communicative competence from 9 to 15 months of age. , 1998, Monographs of the Society for Research in Child Development.

[18]  Terrence J Sejnowski,et al.  Foundations for a New Science of Learning , 2009, Science.

[19]  Chen Yu,et al.  The Role of Embodied Intention in Early Lexical Acquisition , 2005, Cogn. Sci..

[20]  Minoru Asada,et al.  A constructive model for the development of joint attention , 2003, Connect. Sci..

[21]  Hideaki Kuzuoka,et al.  Precision timing in human-robot interaction: coordination of head movement and utterance , 2008, CHI.