Preliminary Study on Real-Time Interactive Virtual Fixture Generation Method for Shared Teleoperation in Unstructured Environments

We present a method for interactively generating virtual fixtures for shared teleoperation in unstructured remote environments. The proposed method allows a human operator to intuitively assign various types of virtual fixtures on-the-fly to provide virtual guidance forces helping the operator to accomplish a given task while minimizing the cognitive workload. The proposed method augments the visual feedback image from the slave’s robot video camera with automatically extracted geometric features (shapes, surfaces, etc.) computed from both depth and color video sensor attached next to the slave robot’s base. The human operator can select a feature on the computer screen which is then automatically associated with a virtual haptic fixture. The performance of the proposed method was evaluated with a peg-in-hole task and the experiment showed improvements in teleoperation performance.

[1]  Yu Liu,et al.  Flexible virtual fixture enhanced by vision and haptics for unstructured environment teleoperation , 2013, 2013 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[2]  Russell H. Taylor,et al.  A constrained optimization approach to virtual fixtures , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Darwin G. Caldwell,et al.  Vision based virtual fixture generation for teleoperated robotic manipulation , 2016, 2016 International Conference on Advanced Robotics and Mechatronics (ICARM).

[4]  Louis B. Rosenberg.  Virtual fixtures as tools to enhance operator performance in telepresence environments , 1993, Other Conferences.

[5]  Marcelo H. Ang,et al.  Flexible virtual fixture interface for path specification in tele-manipulation , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[6]  Louis B. Rosenberg,et al.  Virtual fixtures: Perceptual tools for telerobotic manipulation , 1993, Proceedings of IEEE Virtual Reality Annual International Symposium.

[7]  Louis B. Rosenberg,et al.  The Use of Virtual Fixtures as Perceptual Overlays to Enhance Operator Performance in Remote Environments. , 1992 .

[8]  Jee-Hwan Ryu,et al.  Improving mobile robot bilateral teleoperation by introducing variable force feedback gain , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Allison M. Okamura,et al.  Haptic Virtual Fixtures for Robot-Assisted Manipulation , 2005, ISRR.

[10]  Russell H. Taylor,et al.  Spatial motion constraints in medical robot using virtual fixtures generated by anatomy , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[11]  Peter Kazanzides,et al.  Portability and applicability of virtual fixtures across medical and manufacturing tasks , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[12]  Francisco José Madrid-Cuevas,et al.  Automatic generation and detection of highly reliable fiducial markers under occlusion , 2014, Pattern Recognit..

[13]  Gregory D. Hager,et al.  Vision assisted control for manipulation using virtual fixtures: experiments at macro and micro scales , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[14]  Jee-Hwan Ryu,et al.  A user study of command strategies for mobile robot teleoperation , 2009, Intell. Serv. Robotics.

[15]  Howard Jay Chizeck,et al.  Forbidden-region virtual fixtures from streaming point clouds: Remotely touching and protecting a beating heart , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[16]  Danica Kragic,et al.  Adaptive Virtual Fixtures for Machine-Assisted Teleoperation Tasks , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[17]  Brian L. Davies,et al.  Active Constraints/Virtual Fixtures: A Survey , 2014, IEEE Transactions on Robotics.

[18]  Blake Hannaford,et al.  Time domain passivity control of haptic interfaces , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[19]  Darwin G. Caldwell,et al.  Enhancing bilateral teleoperation using camera-based online virtual fixtures generation , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).