Enabling effective human-robot interaction using perspective-taking in robots

We propose that an important aspect of human-robot interaction is perspective-taking. We show how perspective-taking occurs in a naturalistic environment (astronauts working on a collaborative project) and present a cognitive architecture for performing perspective-taking called Polyscheme. Finally, we show a fully integrated system that instantiates our theoretical framework within a working robot system. Our system successfully solves a series of perspective-taking problems and uses the same frames of references that astronauts do to facilitate collaborative problem solving with a person.

[1]  Alan C. Schultz,et al.  Using a natural language and gesture interface for unmanned vehicles , 2000, Defense, Security, and Sensing.

[2]  A. Newell.  Unified Theories of Cognition , 1990 .

[3]  Alan C. Schultz,et al.  Integrating natural language and gesture in a robotics domain , 1998, Proceedings of the 1998 IEEE International Symposium on Intelligent Control (ISIC) held jointly with IEEE International Symposium on Computational Intelligence in Robotics and Automation (CIRA) Intell.

[4]  David E. Kieras,et al.  An Overview of the EPIC Architecture for Cognition and Performance With Application to Human-Computer Interaction , 1997, Hum. Comput. Interact..

[5]  C. Lebiere,et al.  The Atomic Components of Thought , 1998 .

[6]  R. Shepard,et al.  Mental Rotation of Three-Dimensional Objects , 1971, Science.

[7]  Donald A. Norman,et al.  Things That Make Us Smart: Defending Human Attributes In The Age Of The Machine , 1993 .

[8]  Susan Bell Trickett,et al.  The Relationship Between Spatial Transformations and Iconic Gestures , 2006, Spatial Cogn. Comput..

[9]  J. Gregory Trafton,et al.  Choosing Frames of Referenece: Perspective-Taking in a 2D and 3D Navigational Task , 2004 .

[10]  Lynne E. Parker,et al.  Multi-Robot Systems: From Swarms to Intelligent Automata , 2002, Springer Netherlands.

[11]  J. Gregory Trafton,et al.  Erratum to "Memories for goals: An activation-based model"[Cognitive Science 26 (2002) 39-83] , 2002, Cogn. Sci..

[12]  C. Creider.  Hand and Mind: What Gestures Reveal about Thought , 1994 .

[13]  Magdalena D. Bugajska,et al.  Building a Multimodal Human-Robot Interface , 2001, IEEE Intell. Syst..

[14]  J. Gregory Trafton,et al.  Children and robots learning to play hide and seek , 2006, HRI '06.

[15]  J. Gregory Trafton,et al.  A Cognitive Model for Spatial Perspective taking , 2004, ICCM.

[16]  B. Tversky,et al.  Perspective in Spatial Descriptions , 1996 .

[17]  Alan C. Schultz,et al.  Integrating Exploration, Localization, Navigation and Planning with a Common Representation , 1999, Auton. Robots.

[18]  J. Gregory Trafton,et al.  Integrating cognition, perception and action through mental simulation in robots , 2004, Robotics Auton. Syst..

[19]  Allen Newell,et al.  The logic theory machine-A complex information processing system , 1956, IRE Trans. Inf. Theory.

[20]  Gillian Ku,et al.  The Effects of Perspective-Taking on Prejudice: The Moderating Role of Self-Evaluation , 2004, Personality & social psychology bulletin.

[21]  James R. Wallace,et al.  Spatial Perspective-Taking Errors in Children , 2001, Perceptual and motor skills.

[22]  J. Huttenlocher,et al.  The Coding of Spatial Location in Young Children , 1994, Cognitive Psychology.

[23]  Marjorie Skubic,et al.  Communicating with Teams of Cooperative Robots , 2002 .

[24]  Marjorie Skubic,et al.  Robot navigation using qualitative landmark states from sketched route maps , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[25]  K. A. Ericsson,et al.  Protocol Analysis: Verbal Reports as Data , 1984 .

[26]  S. Goldin-Meadow,et al.  When Gestures and Words Speak Differently , 1997 .

[27]  Clark C. Presson,et al.  The coding and transformation of spatial information , 1979, Cognitive Psychology.

[28]  M. Hegarty,et al.  A dissociation between mental rotation and perspective-taking spatial abilities , 2004 .

[29]  John H. Flavell,et al.  The development of three spatial perspective-taking rules. , 1981 .

[30]  Derek Anderson,et al.  Using a qualitative sketch to control a team of robots , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[31]  Laura A. Carlson-Radvansky,et al.  The Influence of Functional Relations on Spatial Term Selection , 1996 .

[32]  John R. Anderson,et al.  Skill Acquisition and the LISP Tutor , 1989, Cogn. Sci..

[33]  J. Gregory Trafton,et al.  Cognition and Multi-Agent Interaction: Communicating and Collaborating with Robotic Agents , 2005 .

[34]  Fredrik Rehnmark,et al.  Robonaut: A Robot Designed to Work with Humans in Space , 2003, Auton. Robots.

[35]  Willem J. M. Levelt,et al.  Some Perceptual Limitations on Talking About Space , 1984 .

[36]  James F. Allen.  Maintaining knowledge about temporal intervals , 1983, CACM.

[37]  K. A. Ericsson,et al.  Protocol analysis: Verbal reports as data, Rev. ed. , 1993 .

[38]  Paul U. Lee,et al.  Why do speakers mix perspectives? , 1999, Spatial Cogn. Comput..

[39]  Manuela M. Veloso,et al.  Fast and inexpensive color image segmentation for interactive robots , 2000, Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No.00CH37113).

[40]  J. Gregory Trafton,et al.  Turning pictures into numbers: extracting and generating information from complex visualizations , 2000, Int. J. Hum. Comput. Stud..

[41]  Karen Emmorey,et al.  Using space to describe space: Perspective in speech, sign, and gesture , 2000, Spatial Cogn. Comput..

[42]  J. Huttenlocher,et al.  Children's Early Ability to Solve Perspective-Taking Problems. , 1992 .

[43]  J. Fodor.  The Modularity of mind. An essay on faculty psychology , 1986 .

[44]  Ellen Bialystok,et al.  Attentional control in children's metalinguistic performance and measures of field independence. , 1992 .

[45]  Rocky Ross,et al.  Mental models , 2004, SIGA.

[46]  D. Laible,et al.  Attachment and emotional understanding in preschool children. , 1998, Developmental psychology.

[47]  J. Gregory Trafton,et al.  Memory for goals: an activation-based model , 2002, Cogn. Sci..

[48]  Alan C. Schultz,et al.  Goal tracking in a natural language interface: towards achieving adjustable autonomy , 1999, Proceedings 1999 IEEE International Symposium on Computational Intelligence in Robotics and Automation. CIRA'99 (Cat. No.99EX375).

[49]  B. Tversky,et al.  Switching points of view in spatial mental models , 1992, Memory & cognition.

[50]  John R. Anderson,et al.  Serial modules in parallel: the psychological refractory period and perfect time-sharing. , 2001, Psychological review.

[51]  Kenneth Wauchope,et al.  Eucalyptus: Integrating Natural Language Input with a Graphical User Interface , 1994 .

[52]  Allen Newell,et al.  SOAR: An Architecture for General Intelligence , 1987, Artif. Intell..

[53]  John E. Laird,et al.  It knows what you're going to do: adding anticipation to a Quakebot , 2001, AGENTS '01.

[54]  D E Kieras,et al.  A computational theory of executive cognitive processes and multiple-task performance: Part 1. Basic mechanisms. , 1997, Psychological review.

[55]  Laura A. Carlson-Radvansky,et al.  The Influence of Reference Frame Selection on Spatial Template Construction , 1997 .

[56]  B. Tversky,et al.  Spatial mental models derived from survey and route descriptions , 1992 .

[57]  Marjorie Skubic,et al.  Qualitative analysis of sketched route maps: translating a sketch into linguistic descriptions , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[58]  Marvin Minsky,et al.  Polyscheme: a cognitive architecture for integrating multiple representation and inference schemes , 2002 .