A novel augmented reality-based interface for robot path planning

Intuitive and efficient interfaces for robot task planning have been a challenging issue in robotics as it is essential for the prevalence of robots supporting humans in key areas of activities. This paper presents a novel augmented reality (AR) based interface for interactive robot path and end-effector (EE) orientation planning. A number of human-virtual robot interaction methods have been formulated and implemented with respect to the various types of robotic operations needed in different applications. A Euclidean distance-based method is developed to assist the users in the modification of the waypoints so as to update the planned paths and/or orientation profiles within the proposed AR environment. The virtual cues augmented in the real environment can support and enhance human-virtual robot interaction at different stages of the robot tasks planning process. Two case studies are presented to demonstrate the successful implementation of the proposed AR-based interface in planning robot pick-and-place tasks and path following tasks.

[1]  Hironao Yamada,et al.  Haptic interaction in tele-operation control system of construction robot based on virtual reality , 2009, 2009 International Conference on Mechatronics and Automation.

[2]  Andrew Y. C. Nee,et al.  Robot programming using augmented reality: An interactive method for planning collision-free paths , 2009 .

[3]  Xing Zhu,et al.  Hand Posture Recognition in Gesture-Based Human-Robot Interaction , 2006, 2006 1ST IEEE Conference on Industrial Electronics and Applications.

[4]  Pier Paolo Valentini.  Interactive cable harnessing in augmented reality , 2011 .

[5]  Alexander Zelinsky,et al.  Programing by Demonstration: Coping with Suboptimal Teaching Actions , 2003 .

[6]  Chang Hoi Kim,et al.  Preventive maintenance and remote inspection of nuclear power plants using tele-robotics , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).

[7]  Gunther Reinhart,et al.  A programming system for robot-based remote-laser-welding with conventional optics , 2008 .

[8]  Grigore C. Burdea,et al.  Virtual reality and robotics in medicine , 1996, Proceedings 5th IEEE International Workshop on Robot and Human Communication. RO-MAN'96 TSUKUBA.

[9]  Hubert Roth,et al.  Applications of automated guided vehicle (AGV) and industry robots with PMD-camera , 2007 .

[10]  Rüdiger Dillmann,et al.  A comparison of four fast vision based object recognition methods for programming by demonstration applications , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[11]  Rainer Bischoff,et al.  Perspectives on augmented reality based human-robot interaction with industrial robots , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[12]  Aude Billard,et al.  On Learning, Representing, and Generalizing a Task in a Humanoid Robot , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[13]  Ch. Meyer,et al.  An Intuitive Teaching Method for Small and Medium Enterprises , 2006 .

[14]  R A Castillo Cruces,et al.  Improving robot arm control for safe and robust haptic cooperation in orthopaedic procedures , 2007, The international journal of medical robotics + computer assisted surgery : MRCAS.

[15]  Brian Scassellati,et al.  Active vision for sociable robots , 2001, IEEE Trans. Syst. Man Cybern. Part A.

[16]  Wolfram Burgard,et al.  MINERVA: a second-generation museum tour-guide robot , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[17]  R. Dillmann,et al.  Sensor fusion approaches for observation of user actions in programming by demonstration , 2001, Conference Documentation International Conference on Multisensor Fusion and Integration for Intelligent Systems. MFI 2001 (Cat. No.01TH8590).

[18]  J. Norberto Pires.  Robot-by-voice: experiments on commanding an industrial robot using the human voice , 2005, Ind. Robot.

[19]  Ali Sekmen,et al.  Human–robot interaction via voice-controllable intelligent user interface , 2007, Robotica.

[20]  Pier Paolo Valentini,et al.  Interactive virtual assembling in augmented reality , 2009 .

[21]  Yonghua Chen,et al.  Haptic-aided robot path planning based on virtual tele-operation , 2009 .

[22]  Leon lajpah,et al.  Simulation in robotics , 2008 .

[23]  Young-Ho Kim,et al.  Museum tour guide robot with augmented reality , 2010, 2010 16th International Conference on Virtual Systems and Multimedia.

[24]  Wim Meeussen.  Compliant robot motion : from path planning or human demonstration to force controlled task execution , 2006 .

[25]  Marcello Pellicciari,et al.  Hybrid Reconfigurable System design and optimization through virtual prototyping and digital manufacturing tools , 2012 .

[26]  Torgny Brogårdh,et al.  Present and future robot control development - An industrial perspective , 2007, Annu. Rev. Control..

[27]  Abhilash Pandya,et al.  Improved Telemanipulator Navigation During Display-Control Misalignments Using Augmented Reality Cues , 2010, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[28]  Takao Kakizaki,et al.  A multimodal teaching advisor for sensor-enhanced robotic systems in manufacturing , 1998 .

[29]  Michael F. Zäh,et al.  Interactive laser-projection for programming industrial robots , 2006, 2006 IEEE/ACM International Symposium on Mixed and Augmented Reality.

[30]  Naoki Kawakami,et al.  Teleoperation system with haptic feedback for physical interaction with remote environment , 2009, 2009 ICCAS-SICE.

[31]  Giuseppe Di Gironimo,et al.  Innovative assembly process for modular train and feasibility analysis in virtual environment , 2009 .

[32]  Christopher D. Wickens,et al.  A model for types and levels of human interaction with automation , 2000, IEEE Trans. Syst. Man Cybern. Part A.

[33]  J. Norberto Pires,et al.  Using digital pens to program welding tasks , 2007, Ind. Robot.

[34]  Raul Wirz,et al.  A multimodal interface to control a robot arm via the web: a case study on remote programming , 2005, IEEE Transactions on Industrial Electronics.

[35]  Andrew Y. C. Nee,et al.  A novel AR-based robot programming and path planning methodology , 2010 .

[36]  Christiaan J. J. Paredis,et al.  Interactive Multimodal Robot Programming , 2005, Int. J. Robotics Res..

[37]  Stefano Caselli,et al.  Trajectory clustering and stochastic approximation for robot programming by demonstration , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[38]  A. Kazi,et al.  The MORPHA style guide for icon-based programming , 2002, Proceedings. 11th IEEE International Workshop on Robot and Human Interactive Communication.

[39]  Andrew Y. C. Nee,et al.  Haptic-based interactive path planning for a virtual robot arm , 2010 .

[40]  Sebastian Thrun.  Toward a framework for human-robot interaction , 2004 .

[41]  Sebastian Thrun,et al.  A Gesture Based Interface for Human-Robot Interaction , 2000, Auton. Robots.