Coordinating Social Communication in Human-Robot Task Collaborations

ix Chapter 1: Introduction 1 1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1.1 Human-Robot Task Collaboration . . . . . . . . . . . . . . . . . 1 1.1.2 Approach to Coordinating Social Communication . . . . . . . . . 3 1.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.3 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Chapter 2: Background and Related Work 10 2.1 Human-Human Collaboration . . . . . . . . . . . . . . . . . . . . . . . . 10 2.1.1 Perspective-Taking and Theory-of-Mind . . . . . . . . . . . . . . 11 2.1.2 Human Activity Modeling and Recognition . . . . . . . . . . . . 13 2.2 Multi-Robot Coordination and Applicability to the Human-Robot Case 13 2.2.1 Task Classifications . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.3 Human-Robot Collaboration . . . . . . . . . . . . . . . . . . . . . . . . 18 2.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 Chapter 3: Planning of Coordinating Social Communication 22 3.1 Human-Robot Task Representation . . . . . . . . . . . . . . . . . . . . . 25 3.2 Representing and Recognizing Human Activity . . . . . . . . . . . . . . 27 3.3 Communication Planning . . . . . . . . . . . . . . . . . . . . . . . . . . 31 3.4 Extensions of the Approach . . . . . . . . . . . . . . . . . . . . . . . . . 33 3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

[1]  Julie Shah,et al.  Human-Robot Teaming using Shared Mental Models , 2012 .

[2]  David Kirsh,et al.  The Intelligent Use of Space , 1995, Artif. Intell..

[3]  Svetha Venkatesh,et al.  Learning and detecting activities from movement trajectories using the hierarchical hidden Markov model , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[4]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[5]  Cynthia Breazeal,et al.  Improved human-robot team performance using Chaski, A human-inspired plan execution system , 2011, 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[6]  Rainer Stiefelhagen,et al.  Visual recognition of pointing gestures for human-robot interaction , 2007, Image Vis. Comput..

[7]  Bernard Horan,et al.  Supporting Flexible Roles in a Shared Space , 2001, Collaborative Virtual Environments.

[8]  Dana H. Ballard,et al.  Deictic human/robot interaction , 1996, Robotics Auton. Syst..

[9]  Stefanos Nikolaidis,et al.  Human-robot cross-training: Computational formulation, modeling and evaluation of a human team training strategy , 2013, 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[10]  Asli Ozyurek.  Do Speakers Design Their Cospeech Gestures for Their Addressees? The Effects of Addressee Location on Representational Gestures , 2002 .

[11]  Claus Lamm,et al.  The Neural Substrate of Human Empathy: Effects of Perspective-taking and Cognitive Appraisal , 2007, Journal of Cognitive Neuroscience.

[12]  Bilge Mutlu,et al.  Modeling and Evaluating Narrative Gestures for Humanlike Robots , 2013, Robotics: Science and Systems.

[13]  Roberto Cipolla,et al.  Human-robot interface by pointing with uncalibrated stereo vision , 1996, Image Vis. Comput..

[14]  Korten Kamp,et al.  Recognizing and interpreting gestures on a mobile robot , 1996, AAAI 1996.

[15]  Cynthia Breazeal,et al.  Cost-Based Anticipatory Action Selection for Human–Robot Fluency , 2007, IEEE Transactions on Robotics.

[16]  Neil Immerman,et al.  The Complexity of Decentralized Control of Markov Decision Processes , 2000, UAI.

[17]  Michael K. Tanenhaus,et al.  Pragmatic effects on reference resolution in a collaborative task: evidence from eye movements , 2004, Cogn. Sci..

[18]  Steve Whittaker,et al.  Things to Talk About When Talking About Things , 2003, Hum. Comput. Interact..

[19]  M. Matarić,et al.  Fixation behavior in observation and imitation of human movement. , 1998, Brain research. Cognitive brain research.

[20]  J. Burgoon,et al.  Nonverbal Communication , 2018, Dental Abstracts.

[21]  Robert E. Kraut,et al.  Coordination of communication: effects of shared visual context on collaborative work , 2000, CSCW '00.

[22]  S. Baron-Cohen.  The evolution of a theory of mind. , 1999 .

[23]  E. Maris,et al.  Two Sides of the Same Coin , 2010, Psychological science.

[24]  Ferdinando Fornara,et al.  Robots in a domestic setting: a psychological approach , 2005, Universal Access in the Information Society.

[25]  Guy Shani,et al.  A survey of point-based POMDP solvers , 2013, Autonomous Agents and Multi-Agent Systems.

[26]  Irfan A. Essa,et al.  A novel sequence representation for unsupervised analysis of human activities , 2009, Artif. Intell..

[27]  Craig Boutilier,et al.  Planning, Learning and Coordination in Multiagent Decision Processes , 1996, TARK.

[28]  Pattie Maes,et al.  Self-Taught Visually-Guided Pointing for a Humanoid Robot , 1996 .

[29]  Bilge Mutlu,et al.  The repertoire of robot behavior , 2013, HRI 2013.

[30]  Joshua B. Tenenbaum,et al.  Help or Hinder: Bayesian Models of Social Goal Inference , 2009, NIPS.

[31]  Stephanie Rosenthal,et al.  An effective personal mobile robot agent through symbiotic human-robot interaction , 2010, AAMAS.

[32]  Christof Koch,et al.  Attentional Selection for Object Recognition - A Gentle Way , 2002, Biologically Motivated Computer Vision.

[33]  Bilge Mutlu,et al.  Coordination Mechanisms in Human-Robot Collaboration , 2013 .

[34]  Cynthia Breazeal,et al.  An Empirical Analysis of Team Coordination Behaviors and Action Planning With Application to Human–Robot Teaming , 2010, Hum. Factors.

[35]  Candace L. Sidner,et al.  Where to look: a study of human-robot engagement , 2004, IUI '04.

[36]  S P Levine,et al.  The NavChair Assistive Wheelchair Navigation System. , 1999, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[37]  J. Gregory Trafton,et al.  Enabling effective human-robot interaction using perspective-taking in robots , 2005, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[38]  D. Barr,et al.  Taking Perspective in Conversation: The Role of Mutual Knowledge in Comprehension , 2000, Psychological science.

[39]  Andrea Lockerd Thomaz,et al.  Reinforcement Learning with Human Teachers: Evidence of Feedback and Guidance with Implications for Learning Performance , 2006, AAAI.

[40]  Candace L. Sidner,et al.  Attention, Intentions, and the Structure of Discourse , 1986, CL.

[41]  Takayuki Kanda,et al.  Footing in human-robot conversations: How robots might shape participant roles using gaze cues , 2009, 2009 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[42]  A. Bangerter,et al.  Using Pointing and Describing to Achieve Joint Focus of Attention in Dialogue , 2004, Psychological science.

[43]  Makoto Yokoo,et al.  Taming Decentralized POMDPs: Towards Efficient Policy Computation for Multiagent Settings , 2003, IJCAI.

[44]  Raj M. Ratwani,et al.  Integrating vision and audition within a cognitive architecture to track conversations , 2008, 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[45]  B. Scassellati,et al.  Inferring narrative and intention from playground games , 2008, 2008 7th IEEE International Conference on Development and Learning.

[46]  Stefanos Nikolaidis,et al.  Optimization of Temporal Dynamics for Adaptive Human-Robot Interaction in Assembly Manufacturing , 2012, Robotics: Science and Systems.

[47]  Rachel I. Mayberry,et al.  Language and Gesture: Gesture production during stuttered speech: insights into the nature of gesture–speech integration , 2000 .

[48]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[49]  F. Thomas,et al.  The illusion of life : Disney animation , 1981 .

[50]  Max M. Louwerse,et al.  Focusing Attention with Deictic Gestures and Linguistic Expressions , 2005 .

[51]  Robert E. Kraut,et al.  Visual Information as a Conversational Resource in Collaborative Physical Tasks , 2003, Hum. Comput. Interact..

[52]  D. McNeill.  Gesture and Thought , 2005 .

[53]  Penelope Brown,et al.  Politeness: some universals in language usage , 1987 .

[54]  Takayuki Kanda,et al.  Humanlike conversation with gestures and verbal cues based on a three-layer attention-drawing model , 2006, Connect. Sci..

[55]  Siddhartha S. Srinivasa,et al.  Legibility and predictability of robot motion , 2013, 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[56]  Monica N. Nicolescu,et al.  An Architecture for Understanding Intent Using a Novel Hidden Markov Formulation , 2008, Int. J. Humanoid Robotics.

[57]  F. Heider,et al.  An experimental study of apparent behavior , 1944 .

[58]  Manuela M. Veloso,et al.  Teaching collaborative multi-robot tasks through demonstration , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.

[59]  M. Pickering,et al.  Toward a mechanistic psychology of dialogue , 2004, Behavioral and Brain Sciences.

[60]  Daniel M. Oppenheimer,et al.  Accuracy in detecting referents of pointing gestures unaccompanied by language , 2006 .

[61]  M. Tomasello.  Why We Cooperate , 2009 .

[62]  Robert E. Kraut,et al.  The value of shared visual information for task-oriented collaboration , 2006 .

[63]  Cynthia Breazeal,et al.  An Embodied Cognition Approach to Mindreading Skills for Socially Intelligent Robots , 2009, Int. J. Robotics Res..

[64]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[65]  John Lasseter,et al.  Principles of traditional animation applied to 3D computer animation , 1987, SIGGRAPH.

[66]  F. Binkofski,et al.  The mirror neuron system and action recognition , 2004, Brain and Language.

[67]  Brian Scassellati,et al.  Investigating models of social development using a humanoid robot , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[68]  Carl Gutwin,et al.  Where are you pointing?: the accuracy of deictic pointing in CVEs , 2010, CHI.

[69]  Juan Fasola,et al.  A socially assistive robot exercise coach for the elderly , 2013, HRI 2013.

[70]  Takayuki Kanda,et al.  Conversational gaze mechanisms for humanlike robots , 2012, TIIS.

[71]  S. Brennan,et al.  Addressees' needs influence speakers' early syntactic choices , 2002, Psychonomic bulletin & review.

[72]  Jenay M. Beer,et al.  The domesticated robot: Design guidelines for assisting older adults to age in place , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[73]  I. Steiner.  Group process and productivity , 1972 .

[74]  Elizabeth L. Martin,et al.  Designing Synthetic Tasks for Human Factors Research: An Application to Uninhabited Air Vehicles , 1998 .

[75]  Brian Scassellati,et al.  Theory of Mind for a Humanoid Robot , 2002, Auton. Robots.

[76]  Sarit Kraus,et al.  The Evolution of Sharedplans , 1999 .

[77]  Takayuki Kanda,et al.  Pointing to space: modeling of deictic interaction referring to regions , 2010, HRI 2010.

[78]  Stefanie Tellex,et al.  Toward understanding natural language directions , 2010, HRI 2010.