Registration of planar virtual fixtures by using augmented reality with dynamic textures

This paper presents a method to align a virtual plane with a real plane in an augmented reality environment. The method addresses the challenge of configuring a planar virtual fixture during the teleoperation of a robot in a visually challenging environment. The method assists the operator to manually align the virtual plane with the real plane by providing visual cues by dynamically texturing the virtual plane. Following the manual initialization a robust evaluation of the optical flow is used to refine the alignment of the virtual plane. The system combines a stereo camera with a stereo display and generates visual distortions in the displayed images to indicate a misalignment of the virtual plane with the real one. The distortion is computed by displaying warped images from the left camera in the right display and vice versa. Results demonstrate that the method improves the accuracy of the alignment with a real plane that is covered with reflective multi-layer insulation.

[1]  François Chaumette,et al.  Visual servo control. II. Advanced approaches [Tutorial] , 2007, IEEE Robotics & Automation Magazine.

[2]  Russell H. Taylor,et al.  Spatial Motion Constraints for Robot Assisted Suturing Using Virtual Fixtures , 2005, MICCAI.

[3]  I. Du,et al.  Direct Methods , 1998 .

[4]  Peter Kazanzides,et al.  An integrated system for planning, navigation and robotic assistance for skull base surgery , 2008, The international journal of medical robotics + computer assisted surgery : MRCAS.

[5]  Louis B. Rosenberg,et al.  Virtual fixtures: Perceptual tools for telerobotic manipulation , 1993, Proceedings of IEEE Virtual Reality Annual International Symposium.

[6]  Howard Jay Chizeck,et al.  Forbidden-region virtual fixtures from streaming point clouds: Remotely touching and protecting a beating heart , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[7]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[8]  Blake Hannaford,et al.  Using Kinect and a Haptic Interface for Implementation of Real-Time Virtual Fixture , 2011 .

[9]  Dana R. Yoerger,et al.  Navigation and control of the Nereus hybrid underwater vehicle for global ocean science to 10,903 m depth: Preliminary results , 2010, 2010 IEEE International Conference on Robotics and Automation.

[10]  Tsuneo Yoshikawa,et al.  Ground-space bilateral teleoperation of ETS-VII robot arm by direct bilateral coupling under 7-s time delay condition , 2004, IEEE Transactions on Robotics and Automation.

[11]  Gregory D. Hager,et al.  Spatial motion constraints: theory and demonstrations for robot guidance using virtual fixtures , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[12]  Howie Choset,et al.  Path Planning for Robotic Demining: Robust Sensor-Based Coverage of Unstructured Environments and Probabilistic Methods , 2003, Int. J. Robotics Res..

[13]  Wolfram Burgard,et al.  Software Architecture of an Autonomous Robotic System , 2004 .

[14]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.

[15]  Blake Hannaford,et al.  Using Kinect TM and a Haptic Interface for Implementation of Real-Time Virtual Fixtures , 2011 .

[16]  Christopher K. DeBolt,et al.  Test Bed Robot Development for Cooperative Submunitions Clearance , 1999, Int. J. Robotics Res..

[17]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[18]  Allison M. Okamura,et al.  Haptic Virtual Fixtures for Robot-Assisted Manipulation , 2005, ISRR.

[19]  Peter Kazanzides,et al.  An open-source research kit for the da Vinci® Surgical System , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[20]  Thomas B. Sheridan,et al.  Space teleoperation through time delay: review and prognosis , 1993, IEEE Trans. Robotics Autom..

[21]  Peter Kazanzides,et al.  Augmented reality environment with virtual fixtures for robotic telemanipulation in space , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[22]  Peter Kazanzides,et al.  Surgical and interventional robotics: part III [Tutorial] , 2008, IEEE Robotics Autom. Mag..

[23]  S. Hutchinson,et al.  Visual Servo Control Part II : Advanced Approaches , 2007 .

[24]  Simon Baker,et al.  Lucas-Kanade 20 Years On: A Unifying Framework , 2004, International Journal of Computer Vision.

[25]  Selim Benhimane,et al.  Homography-based 2D Visual Tracking and Servoing , 2007, Int. J. Robotics Res..

[26]  Peter Kazanzides,et al.  A component-based architecture for flexible integration of robotic systems , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.