Novel AR-based interface for human-robot interaction and visualization

Intuitive and efficient interfaces for human-robot interaction (HRI) has been a challenging issue in robotics as it is essential for the prevalence of robots supporting humans in key areas of activities. This paper presents a novel augmented reality (AR) based interface to facilitate human-virtual robot interaction. A number of human-virtual robot interaction methods have been formulated and implemented with respect to the various types of operations needed in different robotic applications. A Euclidean distance-based method is developed to assist the users in the interaction with the virtual robot and the spatial entities in an AR environment. A monitor-based visualization mode is adopted as it enables the users to perceive the virtual contents associated with different interaction methods, and the virtual content augmented in the real environment is informative and useful to the users during their interaction with the virtual robot. Case researches are presented to demonstrate the successful implementation of the AR-based HRI interface in planning robot pick-and-place operations and path following operations.

[1]  J. Norberto Pires,et al.  Using digital pens to program welding tasks , 2007, Ind. Robot.

[2]  Hubert Roth,et al.  Applications of automated guided vehicle (AGV) and industry robots with PMD-camera , 2007 .

[3]  Andrew Y. C. Nee,et al.  Robot programming using augmented reality: An interactive method for planning collision-free paths , 2009 .

[4]  Ch. Meyer,et al.  An Intuitive Teaching Method for Small and Medium Enterprises , 2006 .

[5]  Leon lajpah,et al.  Simulation in robotics , 2008 .

[6]  Young-Ho Kim,et al.  Museum tour guide robot with augmented reality , 2010, 2010 16th International Conference on Virtual Systems and Multimedia.

[7]  Christopher D. Wickens,et al.  A model for types and levels of human interaction with automation , 2000, IEEE Trans. Syst. Man Cybern. Part A.

[8]  Andrew Y. C. Nee,et al.  Interactive robot trajectory planning and simulation using Augmented Reality , 2012 .

[9]  Sebastian Thrun,et al.  A Gesture Based Interface for Human-Robot Interaction , 2000, Auton. Robots.

[10]  Shiqi Li,et al.  Predictive display and interaction of telerobots based on augmented reality , 2006, Robotica.

[11]  A. Kazi,et al.  The MORPHA style guide for icon-based programming , 2002, Proceedings. 11th IEEE International Workshop on Robot and Human Interactive Communication.

[12]  Gunther Reinhart,et al.  A programming system for robot-based remote-laser-welding with conventional optics , 2008 .

[13]  Robin R. Murphy,et al.  Human-robot interaction in rescue robotics , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[14]  Ali Sekmen,et al.  Human–robot interaction via voice-controllable intelligent user interface , 2007, Robotica.

[15]  Michael F. Zäh,et al.  Interactive laser-projection for programming industrial robots , 2006, 2006 IEEE/ACM International Symposium on Mixed and Augmented Reality.

[16]  J. Geoffrey Chase,et al.  Evaluating the Augmented Reality Human-Robot Collaboration System , 2008, 2008 15th International Conference on Mechatronics and Machine Vision in Practice.

[17]  Paul Milgram,et al.  Telerobotic control with stereoscopic augmented reality , 1996, Electronic Imaging.

[18]  Naoki Kawakami,et al.  Teleoperation system with haptic feedback for physical interaction with remote environment , 2009, 2009 ICCAS-SICE.

[19]  J. Norberto Pires.  Robot-by-voice: experiments on commanding an industrial robot using the human voice , 2005, Ind. Robot.

[20]  Abhilash Pandya,et al.  Improved Telemanipulator Navigation During Display-Control Misalignments Using Augmented Reality Cues , 2010, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[21]  Takao Kakizaki,et al.  A multimodal teaching advisor for sensor-enhanced robotic systems in manufacturing , 1998 .

[22]  Ahmad B. Rad,et al.  Autonomous humanoid robot navigation using augmented reality technique , 2011, 2011 IEEE International Conference on Mechatronics.

[23]  Julie A. Adams,et al.  Critical Considerations for Human-Robot Interface Development , 2002 .

[24]  Sebastian Thrun.  Toward a framework for human-robot interaction , 2004 .

[25]  Mica R. Endsley,et al.  Direct Measurement of Situation Awareness: Validity and Use of SAGAT , 2000 .

[26]  Grigore C. Burdea,et al.  Virtual reality and robotics in medicine , 1996, Proceedings 5th IEEE International Workshop on Robot and Human Communication. RO-MAN'96 TSUKUBA.

[27]  Clint Heyer.  Human-robot interaction and future industrial robotics applications , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[28]  Jean Scholtz,et al.  Common metrics for human-robot interaction , 2006, HRI '06.

[29]  Wim Meeussen.  Compliant robot motion : from path planning or human demonstration to force controlled task execution , 2006 .

[30]  Xing Zhu,et al.  Hand Posture Recognition in Gesture-Based Human-Robot Interaction , 2006, 2006 1ST IEEE Conference on Industrial Electronics and Applications.

[31]  Alexander Zelinsky,et al.  Programing by Demonstration: Coping with Suboptimal Teaching Actions , 2003 .

[32]  Rüdiger Dillmann,et al.  A comparison of four fast vision based object recognition methods for programming by demonstration applications , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[33]  R A Castillo Cruces,et al.  Improving robot arm control for safe and robust haptic cooperation in orthopaedic procedures , 2007, The international journal of medical robotics + computer assisted surgery : MRCAS.

[34]  Wolfram Burgard,et al.  MINERVA: a second-generation museum tour-guide robot , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[35]  Raul Wirz,et al.  A multimodal interface to control a robot arm via the web: a case study on remote programming , 2005, IEEE Transactions on Industrial Electronics.

[36]  Michael A. Goodrich,et al.  Human-Robot Interaction: A Survey , 2008, Found. Trends Hum. Comput. Interact..

[37]  Andrew Y. C. Nee,et al.  A novel AR-based robot programming and path planning methodology , 2010 .

[38]  Oussama Khatib,et al.  Whole-Body Dynamic Behavior and Control of Human-like Robots , 2004, Int. J. Humanoid Robotics.

[39]  Mica R. Endsley,et al.  Theoretical Underpinnings of Situation Awareness, A Critical Review , 2000 .

[40]  Daniel J. Garland,et al.  Situation Awareness Analysis and Measurement , 2009 .

[41]  Brian Scassellati,et al.  Active vision for sociable robots , 2001, IEEE Trans. Syst. Man Cybern. Part A.

[42]  Christiaan J. J. Paredis,et al.  Interactive Multimodal Robot Programming , 2005, Int. J. Robotics Res..

[43]  Stefano Caselli,et al.  Trajectory clustering and stochastic approximation for robot programming by demonstration , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.