Markerless Augmented Reality for Robots in Unprepared Environments

Augmented Reality (AR) can assist humans in understanding complex robot information, and improve Human and Robot Interaction (HRI). However, many restrictions are imposed by the underlying technology used and thus have limited current AR systems to operate in controlled or modified robot environments. This hinders the wide spread use of AR for different robot applications. This paper presents a markerless AR system that combines recent tracking and detection techniques for AR visualisation of robot task relevant information. We employ natural feature tracking techniques to compute the camera pose for accurate registration of virtual objects. Automatic relocalisation of the camera pose is achieved using a planar object detection algorithm which recovers from tracking failures. Experiments using a camera mounted on a mobile ground robot demonstrated accurate tracking and successful recovery of planar features in an unprepared indoor environment.

[1]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Rainer Lienhart,et al.  An extended set of Haar-like features for rapid object detection , 2002, Proceedings. International Conference on Image Processing.

[3]  Jeffrey E. Boyd,et al.  Implementing Bubblegrams: The Use of Haar-Like Features for Human-Robot Interaction , 2006, 2006 IEEE International Conference on Automation Science and Engineering.

[4]  Yuichi Ohta,et al.  Mixed Reality: Merging Real and Virtual Worlds , 1999 .

[5]  Éric Marchand,et al.  ViSP for visual servoing: a generic software platform with a wide class of robot control skills , 2005, IEEE Robotics & Automation Magazine.

[6]  J. Meigs,et al.  WHO Technical Report , 1954, The Yale Journal of Biology and Medicine.

[7]  Bruce A. MacDonald,et al.  Augmented reality visualisation for player , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[8]  Marie-Odile Berger,et al.  Pose Estimation for Planar Structures , 2002, IEEE Computer Graphics and Applications.

[9]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[10]  Andrew W. Fitzgibbon,et al.  Markerless tracking using planar structures in the scene , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[11]  田村 秀行,et al.  Mixed reality : merging real and virtual worlds , 1999 .

[12]  Manolis I. A. Lourakis,et al.  Vision-based camera motion recovery for augmented reality , 2004, Proceedings Computer Graphics International, 2004..

[13]  Éric Marchand,et al.  Robust model-based tracking for robot vision , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[14]  Hirokazu Kato,et al.  Marker tracking and HMD calibration for a video-based augmented reality conferencing system , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[15]  Gregory M. P. O'Hare,et al.  Using Mixed Reality Agents as Social Interfaces for Robots , 2007, RO-MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication.

[16]  Larry S. Davis,et al.  Model-based object pose in 25 lines of code , 1992, International Journal of Computer Vision.

[17]  Cordelia Schmid,et al.  A Comparison of Affine Region Detectors , 2005, International Journal of Computer Vision.

[18]  Dieter Schmalstieg,et al.  Artoolkitplus for pose tracking on mobile devices , 2007 .

[19]  David W. Murray,et al.  Full-3D Edge Tracking with a Particle Filter , 2006, BMVC.

[20]  Burkhard Wünsche,et al.  Markerless Augmented Reality for Robotic Helicoptor Applications , 2008, RobVis.

[21]  Dieter Schmalstieg,et al.  ARToolKitPlus for Pose Trackin on Mobile Devices , 2007 .

[22]  Vincent Lepetit,et al.  Fast Keypoint Recognition in Ten Lines of Code , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Fumitoshi Matsuno,et al.  Time Follower's Vision: a teleoperation interface with past images , 2005, IEEE Computer Graphics and Applications.

[24]  David G. Lowe,et al.  Scene modelling, recognition and tracking with invariant image features , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[25]  Bruce A. MacDonald,et al.  Towards Improving SLAM Algorithm Development using Augmented Reality , 2007 .

[26]  Olivier Stasse,et al.  MonoSLAM: Real-Time Single Camera SLAM , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Satoshi Kagami,et al.  Augmented Reality for Robot Development and Experimentation , 2005 .