Forbidden region virtual fixtures from streaming point clouds

Forbidden region virtual fixtures protect objects from unwanted contact with a robot. In this paper, we propose a method for creating forbidden region haptic virtual fixtures for teleoperation from streaming point clouds obtained by an RGB-D camera. Upon violating the protected area, the operator receives force feedback that opposes motion inside the forbidden region. Three architectures for creating virtual fixtures are presented and their advantages and disadvantages are described. The proposed methods have the ability to implement constraints and can handle dynamic environments in real-time. The effectiveness of the methods is demonstrated in experiments with a surgical robot. Graphical Abstract

[1]  Gregory D. Hager,et al.  Vision-Based Assistance for Ophthalmic Micro-Surgery , 2004, MICCAI.

[2]  J. Edward Colgate,et al.  Cobot architecture , 2001, IEEE Trans. Robotics Autom..

[3]  Russell H. Taylor,et al.  A constrained optimization approach to virtual fixtures , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[4]  Brian L. Davies,et al.  Active-Constraint Robotics for Surgery , 2006, Proceedings of the IEEE.

[5]  Gregory D. Hager,et al.  Spatial motion constraints: theory and demonstrations for robot guidance using virtual fixtures , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[6]  Allison M. Okamura,et al.  Speed-Accuracy Characteristics of Human-Machine Cooperative Manipulation Using Virtual Fixtures With Variable Admittance , 2004, Hum. Factors.

[7]  Shahram Payandeh,et al.  On application of virtual fixtures as an aid for telemanipulation and training , 2002, Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002.

[8]  Gregory D. Hager,et al.  Handheld micromanipulation with vision-based virtual fixtures , 2011, 2011 IEEE International Conference on Robotics and Automation.

[9]  Blake Hannaford,et al.  Raven-II: An Open Platform for Surgical Robotics Research , 2013, IEEE Transactions on Biomedical Engineering.

[10]  John Kenneth Salisbury,et al.  A constraint-based god-object method for haptic display , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[11]  Timothy N Judkins,et al.  Augmented reality and haptic interfaces for robot‐assisted surgery , 2012, The international journal of medical robotics + computer assisted surgery : MRCAS.

[12]  D. Yuh,et al.  Effect of sensory substitution on suture-manipulation forces for robotic surgical systems. , 2005, The Journal of thoracic and cardiovascular surgery.

[13]  Kostas E. Bekris,et al.  Visual and force-feedback guidance for robot-assisted interventions in the beating heart with real-time MRI , 2012, 2012 IEEE International Conference on Robotics and Automation.

[14]  Cagatay Basdogan,et al.  Haptic Rendering in Virtual Environments , 2002 .

[15]  Howard Jay Chizeck,et al.  Proxy method for fast haptic rendering from time varying point clouds , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[16]  Louis B. Rosenberg.  The use of Virtual Fixtures to Enhance Operator Performance in Time Delayed Teleoperation. , 1993 .

[17]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[18]  Allison M. Okamura,et al.  Virtual fixtures for bilateral telemanipulation , 2006 .

[19]  Russell H. Taylor,et al.  Spatial Motion Constraints Using Virtual Fixtures Generated by Anatomy , 2007, IEEE Transactions on Robotics.

[20]  Pieter Abbeel,et al.  Tracking deformable objects with point clouds , 2013, 2013 IEEE International Conference on Robotics and Automation.

[21]  David Kim,et al.  Shake'n'sense: reducing interference for overlapping structured light depth cameras , 2012, CHI.

[22]  Howard Jay Chizeck,et al.  A Proxy Method for Real-Time 3-DOF Haptic Rendering of Streaming Point Cloud Data , 2013, IEEE Transactions on Haptics.

[23]  Guang-Zhong Yang,et al.  Dynamic Active Constraints for Hyper-Redundant Flexible Robots , 2009, MICCAI.

[24]  Jaesoon Choi,et al.  Haptic virtual fixture for robotic cardiac catheter navigation. , 2011, Artificial organs.

[25]  J. Edward Colgate,et al.  Issues in the haptic display of tool use , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[26]  Howard Jay Chizeck,et al.  A method for constraint-based six degree-of-freedom haptic interaction with streaming point clouds , 2013, 2013 IEEE International Conference on Robotics and Automation.

[27]  Robert D. Howe,et al.  Virtual Fixtures for Robotic Cardiac Surgery , 2001, MICCAI.

[28]  Louis B. Rosenberg,et al.  Virtual fixtures: Perceptual tools for telerobotic manipulation , 1993, Proceedings of IEEE Virtual Reality Annual International Symposium.

[29]  Blake Hannaford,et al.  Plugfest 2009: Global interoperability in Telerobotics and telemedicine , 2010, 2010 IEEE International Conference on Robotics and Automation.

[30]  Jing Ren,et al.  Dynamic 3-D Virtual Fixtures for Minimally Invasive Beating Heart Procedures , 2008, IEEE Transactions on Medical Imaging.

[31]  Alexei Sourin,et al.  Haptic interaction with 2D images , 2011, VRCAI '11.

[32]  John Kenneth Salisbury,et al.  Point clouds can be represented as implicit surfaces for constraint-based haptic rendering , 2012, 2012 IEEE International Conference on Robotics and Automation.

[33]  Gregory D. Hager,et al.  Vision-assisted control for manipulation using virtual fixtures , 2001, IEEE Transactions on Robotics.

[34]  Gregory D. Hager,et al.  Human-Machine Collaborative Systems for Microsurgical Applications , 2005, Int. J. Robotics Res..

[35]  Howard Jay Chizeck,et al.  Forbidden-region virtual fixtures from streaming point clouds: Remotely touching and protecting a beating heart , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.