Autonomous Data-Driven Manipulation of Unknown Anisotropic Deformable Tissues Using Unmodelled Continuum Manipulators

We present an autonomous manipulation approach for tissues with anisotropic deformation behavior using a continuum manipulator. The key feature of our vision-based study is an online learning and estimation method, which makes its implementation independent of any prior knowledge about the deformation behavior of the tissue and continuum manipulator as well as calibration of the vision system with respect to the robot. This important feature addresses the difficulty of using model-based control approaches in deformation control of a continuum manipulator manipulating an unknown deformable tissue. We evaluated the performance and robustness of our method in three different experiments using the da Vinci Research Kit coupled with a 5 mm instrument that has a 4-degree-of-freedom snake-like wrist. These experiments simulated situations that occur in various surgical schemes and verified the adaptability, learning capability, and accuracy of the proposed method.

[1]  C. G. Broyden.  A Class of Methods for Solving Nonlinear Simultaneous Equations , 1965 .

[2]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[3]  Yunhui Liu,et al.  On the visual deformation servoing of compliant objects: Uncalibrated control methods and experiments , 2014, Int. J. Robotics Res..

[4]  Zerui Wang,et al.  Toward Semi-autonomous Cryoablation of Kidney Tumors via Model-Independent Deformable Tissue Manipulation Technique , 2018, Annals of Biomedical Engineering.

[5]  Long Wang,et al.  Medical Technologies and Challenges of Robot Assisted Minimally Invasive Intervention and Diagnostics , 2018, Annu. Rev. Control. Robotics Auton. Syst..

[6]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[7]  Nabil Simaan,et al.  Kinematics-Based Detection and Localization of Contacts Along Multisegment Continuum Robots , 2012, IEEE Transactions on Robotics.

[8]  Guang-Zhong Yang,et al.  Emerging Robotic Platforms for Minimally Invasive Surgery , 2013, IEEE Reviews in Biomedical Engineering.

[9]  Michael C. Yip,et al.  Model-Less Hybrid Position/Force Control: A Minimalist Approach for Continuum Manipulators in Unknown, Constrained Environments , 2016, IEEE Robotics and Automation Letters.

[10]  Mamoru Mitsuishi,et al.  Online Trajectory Planning in Dynamic Environments for Surgical Task Automation , 2014, Robotics: Science and Systems.

[11]  Russell H. Taylor,et al.  Spatial Motion Constraints Using Virtual Fixtures Generated by Anatomy , 2007, IEEE Transactions on Robotics.

[12]  Simon Baker,et al.  Lucas-Kanade 20 Years On: A Unifying Framework , 2004, International Journal of Computer Vision.

[13]  Robert J. Webster,et al.  A Telerobotic System for Transnasal Surgery , 2014, IEEE/ASME Transactions on Mechatronics.

[14]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[15]  Rochdi Merzouki,et al.  Adaptive Neural Network Control of a Compact Bionic Handling Arm , 2015, IEEE/ASME Transactions on Mechatronics.

[16]  Jin Seob Kim,et al.  Nonholonomic Modeling of Needle Steering , 2006, Int. J. Robotics Res..

[17]  Ryan J. Murphy,et al.  Design and kinematic characterization of a surgical manipulator with a focus on treating osteolysis , 2013, Robotica.

[18]  Peter Kazanzides,et al.  An open-source research kit for the da Vinci® Surgical System , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Ryan J. Murphy,et al.  Toward robot-assisted hard osteolytic lesion treatment using a continuum manipulator , 2016, EMBC.

[20]  Nicolas Andreff,et al.  Eye-in-Hand Visual Servoing of Concentric Tube Robots , 2018, IEEE Robotics and Automation Letters.

[21]  James F. O'Brien,et al.  Interactive simulation of surgical needle insertion and steering , 2009, ACM Trans. Graph..

[22]  Zerui Wang,et al.  A Robust Data-Driven Approach for Online Learning and Manipulation of Unmodeled 3-D Heterogeneous Compliant Objects , 2018, IEEE Robotics and Automation Letters.

[23]  CianchettiMatteo,et al.  Learning Closed Loop Kinematic Controllers for Continuum Manipulators in Unstructured Environments , 2017 .

[24]  Ryan S. Decker,et al.  Supervised autonomous robotic soft tissue surgery , 2016, Science Translational Medicine.

[25]  Russell H. Taylor,et al.  Development and Experimental Evaluation of Concurrent Control of a Robotic Arm and Continuum Manipulator for Osteolytic Lesion Treatment , 2017, IEEE Robotics and Automation Letters.

[26]  Pierre E. Dupont,et al.  Percutaneous intracardiac beating-heart surgery using metal MEMS tissue approximation tools , 2012, Int. J. Robotics Res..

[27]  Ian D. Walker,et al.  A Neural Network Controller for Continuum Robots , 2007, IEEE Transactions on Robotics.

[28]  Russell H. Taylor,et al.  A Curved-Drilling Approach in Core Decompression of the Femoral Head Osteonecrosis Using a Continuum Manipulator , 2017, IEEE Robotics and Automation Letters.

[29]  Nilanjan Sarkar,et al.  Robot assisted real-time tumor manipulation for breast biopsy , 2008, 2008 IEEE International Conference on Robotics and Automation.

[30]  Russell H. Taylor,et al.  Integration of a Snake-like Dexterous Manipulator for Head and Neck Surgery with the da Vinci Research Kit. , 2017 .

[31]  Mehran Armand,et al.  A Convex Optimization Framework for Constrained Concurrent Motion Control of a Hybrid Redundant Surgical System , 2018, 2018 Annual American Control Conference (ACC).

[32]  Mehran Armand,et al.  Inroads Toward Robot-Assisted Internal Fixation of Bone Fractures Using a Bendable Medical Screw and the Curved Drilling Technique , 2018, 2018 7th IEEE International Conference on Biomedical Robotics and Biomechatronics (Biorob).

[33]  Michael C. Yip,et al.  Model-Less Feedback Control of Continuum Manipulators in Constrained Environments , 2014, IEEE Transactions on Robotics.

[34]  Tsuneo Yoshikawa,et al.  Manipulability of Robotic Mechanisms , 1985 .

[35]  Yunhui Liu,et al.  Automatic 3-D Manipulation of Soft Objects by Robotic Arms With an Adaptive Deformation Model , 2016, IEEE Transactions on Robotics.

[36]  Cecilia Laschi,et al.  Neural Network and Jacobian Method for Solving the Inverse Statics of a Cable-Driven Soft Arm With Nonconstant Curvature , 2015, IEEE Transactions on Robotics.

[37]  Pieter Abbeel,et al.  Learning by observation for surgical subtasks: Multilateral cutting of 3D viscoelastic and 2D Orthotropic Tissue Phantoms , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).