4 Augmented Reality for Human-Robot Collaboration

Although robotics is well established as a research field, there has been relatively little work on human-robot collaboration. This type of collaboration is going to become an increasingly important issue as robots work ever more closely with humans. For example, in space exploration, recent research has pointed out that to reduce human workload, costs, fatigue driven error and risk, intelligent robotic systems will need to be a significant part of mission design (Fong and Nourbakhsh 2005). Fong and Nourbakhsh also observe that scant attention has been paid to joint human-robot teams, and that making human-robot collaboration natural and efficient is crucial to future space exploration. NASA’s vision for space exploration stresses the cultivation of human-robotic systems (NASA 2004). In addition, companies such as Honda (Honda 2007), Toyota (Toyota 2007) and Sony (Sony 2007) are interested in developing consumer robots that interact with humans in the home and workplace. Finally, the Cogniron project (COGNIRON 2007), MIT Media lab (Hoffmann and Breazeal 2004) and the Mitsubishi Electric Research Laboratories (Sidner and Lee 2005), among others, are currently conducting research in human-robot interaction (HRI). HRI has become a research field in its own right, as shown by the 2006 inaugural conference for HRI with the theme Toward Human-Robot Collaboration (HRI2006 2006). Research into human-human communication can be used as a starting point in developing a robust human-robot collaboration system. Previous research with humans has shown that grounding, situational awareness, a common frame of reference and spatial referencing are vital in effective communication. Clearly, there is a growing need for research on humanrobot collaboration and models of communication between humans and robotic systems. Augmented Reality (AR) is a technology for overlaying three-dimensional virtual graphics onto the users view of the real world. It also allows for real time interaction with these virtual graphics, enabling a user to reach into the augmented world and manipulate it directly. Augmented Reality could be used to overlay virtual imagery on a real robot and so display the internal state and intentions of the robot. Thus AR can bridge the divide between human and robotic systems and could enable effective human-robot collaboration. In this chapter an overview of models of human-human collaboration is provided and how these models could be used to develop a model for human-robot collaboration is investigated. The field of human-robot interaction is reviewed and how it fits into a model

[1]  A. Kendon Some functions of gaze-direction in social interaction. , 1967, Acta psychologica.

[2]  M. Argyle The Psychology of Interpersonal Behaviour , 1967 .

[3]  H. H. Clark,et al.  Referring as a collaborative process , 1986, Cognition.

[4]  Shumin Zhai,et al.  Applications of augmented reality for human-robot communication , 1993, Proceedings of 1993 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '93).

[5]  C. Creider Hand and Mind: What Gestures Reveal about Thought , 1994 .

[6]  P. Milgram,et al.  A Taxonomy of Mixed Reality Visual Displays , 1994 .

[7]  Clifford Nass,et al.  Computers are social actors , 1994, CHI '94.

[8]  Kenji Sakamoto,et al.  Multimodal Interaction in Human Communication , 1995, IEICE Trans. Inf. Syst..

[9]  Y. Inagaki,et al.  Behavior-based intention inference for intelligent robots cooperating with human , 1995, Proceedings of 1995 IEEE International Conference on Fuzzy Systems..

[10]  Lefteri H. Tsoukalas,et al.  Modeling instructible robots for waste disposal applications , 1996, Proceedings IEEE International Joint Symposia on Intelligence and Systems.

[11]  Mark Billinghurst,et al.  Wearable computers for three dimensional CSCW , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[12]  K. Suzuki,et al.  Development of a human and robot collaborative system for inspecting patrol of nuclear power plants , 1997, Proceedings 6th IEEE International Workshop on Robot and Human Communication. RO-MAN'97 SENDAI.

[13]  Mark Billinghurst,et al.  Spatial Information Displays on a Wearable Computer , 1998, IEEE Computer Graphics and Applications.

[14]  Koji Shibuya,et al.  Design and control of mobile manipulation system for human symbiotic humanoid: Hadaly-2 , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[15]  Paul U. Lee,et al.  Why do speakers mix perspectives? , 1999, Spatial Cogn. Comput..

[16]  K. Chang,et al.  Embodiment in conversational interfaces: Rea , 1999, CHI '99.

[17]  Kunikatsu Takase,et al.  Remote collaboration through time delay in multiple teleoperation , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).

[18]  Illah R. Nourbakhsh,et al.  An Affective Mobile Robot Educator with a Full-Time Job , 1999, Artif. Intell..

[19]  Steven K. Feiner,et al.  Information filtering for mobile augmented reality , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[20]  Tetsuo Sawaragi,et al.  Naturalistic human-robot collaboration based upon mixed-initiative interactions in teleoperating environment , 2000, Smc 2000 conference proceedings. 2000 ieee international conference on systems, man and cybernetics. 'cybernetics evolving to systems, humans, organizations, and their complex interactions' (cat. no.0.

[21]  Ivan Poupyrev,et al.  Mixing realities in Shared Space: an augmented reality interface for collaborative computing , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[22]  Ivan Poupyrev,et al.  The MagicBook: a transitional AR interface , 2001, Comput. Graph..

[23]  Ronald Azuma,et al.  Recent Advances in Augmented Reality , 2001, IEEE Computer Graphics and Applications.

[24]  Nak Young Chong,et al.  Exploring interactive simulator in collaborative multi-site teleoperation , 2001, Proceedings 10th IEEE International Workshop on Robot and Human Interactive Communication. ROMAN 2001 (Cat. No.01TH8591).

[25]  Brian Scassellati,et al.  Active vision for sociable robots , 2001, IEEE Trans. Syst. Man Cybern. Part A.

[26]  Carlos Balaguer,et al.  Active human-mobile manipulator cooperation through intention recognition , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[27]  Yukiko I. Nakano,et al.  Non-Verbal Cues for Discourse Structure , 2022 .

[28]  Jean Scholtz,et al.  Human-Robot Interactions: Creating Synergistic Cyber Forces , 2002 .

[29]  Tetsuo Ono,et al.  Development and evaluation of an interactive humanoid robot "Robovie" , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[30]  Hirokazu Kato,et al.  Interactive Theatre Experience in Embodied + Wearable Mixed Reality Space , 2002, ISMAR.

[31]  Marjorie Skubic,et al.  Using spatial language in a human-robot dialog , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[32]  Anoop Gupta,et al.  Communication behaviors of co-located users in collaborative AR interfaces , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[33]  Kerstin Fischer,et al.  Spatial Strategies in Human-Robot Communication , 2002, Künstliche Intell..

[34]  Charles E. Thorpe,et al.  Robot as Partner: Vehicle Teleoperation with Collaborative Control , 2002 .

[35]  Matthias Denecke Rapid Prototyping for Spoken Dialogue Systems , 2002, COLING.

[36]  Hirokazu Kato,et al.  3-D live: real time interaction for mixed reality , 2002, CSCW '02.

[37]  James C. Maida,et al.  Utilization of the Space Vision System as an Augmented Reality System For Mission Operations , 2003 .

[38]  Wei Liu,et al.  Human Pacman: A Mobile Entertainment System with Ubiquitous Computing and Tangible Interaction over a Wide Outdoor Area , 2003, Mobile HCI.

[39]  Christian Faubel,et al.  Anthropomorphism as a pervasive design concept for a robotic assistant , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[40]  Philip R. Cohen,et al.  Referring as a Collaborative Process , 2003 .

[41]  Dieter Schmalstieg,et al.  Collaborative Augmented Reality for Outdoor Navigation and Information Browsing , 2003 .

[42]  Terrence Fong,et al.  Multi-robot remote driving with collaborative control , 2003, IEEE Trans. Ind. Electron..

[43]  Jean Scholtz,et al.  Theory and evaluation of human robot interactions , 2003, 36th Annual Hawaii International Conference on System Sciences, 2003. Proceedings of the.

[44]  Robert E. Kraut,et al.  Effects of head-mounted and scene-oriented video systems on remote collaboration on physical tasks , 2003, CHI '03.

[45]  Candace L. Sidner,et al.  Engagement rules for human-robot collaborative interactions , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[46]  Terrence Fong,et al.  Robot, asker of questions , 2003, Robotics Auton. Syst..

[47]  Yael Edan,et al.  Human-robot collaboration for improved target recognition of agricultural robots , 2003, Ind. Robot.

[48]  Hideaki Kuzuoka,et al.  Dual ecologies of robot as communication media: thoughts on coordinating orientations and projectability , 2004, CHI.

[49]  Cory D. Kidd,et al.  HUMANOID ROBOTS AS COOPERATIVE PARTNERS FOR PEOPLE , 2004 .

[50]  Anders Green,et al.  Involving users in the design of a mobile office robot , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[51]  Cynthia Breazeal,et al.  Social interactions in HRI: the robot view , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[52]  Rüdiger Dillmann,et al.  Using augmented reality to interact with an autonomous mobile platform , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[53]  Robert O. Ambrose,et al.  Cooperative manipulation between humans and teleoperated agents , 2004, 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2004. HAPTICS '04. Proceedings..

[54]  Jean Scholtz,et al.  Beyond Usability Evaluation: Analysis of Human-Robot Interaction at a Major Robotics Competition , 2004, Hum. Comput. Interact..

[55]  Cynthia Breazeal,et al.  Robots that Work in Collaboration with People , 2004, AAAI Technical Report.

[56]  Robin R. Murphy,et al.  Human-robot interaction in rescue robotics , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[57]  Nilanjan Sarkar,et al.  Anxiety detecting robotic system – towards implicit human-robot collaboration , 2004, Robotica.

[58]  Marjorie Skubic,et al.  Spatial language for human-robot dialogs , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[59]  Deb Roy,et al.  Mental imagery for a conversational robot , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[60]  Ruediger Dillmann,et al.  Sharing skills: using augmented reality for human-robot collaboration , 2004, IS&T/SPIE Electronic Imaging.

[61]  Mark Billinghurst,et al.  Designing augmented reality interfaces , 2005, COMG.

[62]  Illah R. Nourbakhsh,et al.  Interaction challenges in human-robot space exploration , 2005, INTR.

[63]  Illah R. Nourbakhsh,et al.  Human-robot teaming for search and rescue , 2005, IEEE Pervasive Computing.

[64]  Candace L. Sidner,et al.  Robots as laboratory hosts , 2005, INTR.

[65]  James C. Maida,et al.  Enhanced Lighting Techniques and Augmented Reality to Improve Human Task Performance , 2005 .

[66]  Bruce A. MacDonald,et al.  Developer oriented visualisation of a robot program , 2006, HRI '06.

[67]  Henrik I. Christensen,et al.  Clarification dialogues in human-augmented mapping , 2006, HRI '06.

[68]  Terrence Fong,et al.  The human-robot interaction operating system , 2006, HRI '06.

[69]  Michael A. Goodrich,et al.  Comparing Situation Awareness for Two Unmanned Aerial Vehicle Human Interface Approaches , 2006 .