Laparoscope arm automatic positioning for robot-assisted surgery based on reinforcement learning

Abstract. Compared with the traditional laparoscopic surgery, the preoperative planning of robot-assisted laparoscopic surgery is more complex and essential. Through the analysis of the surgical procedures and surgical environment, the laparoscope arm preoperative planning algorithm based on the artificial pneumoperitoneum model, lesion parametrization model is proposed, which ensures that the laparoscope arm satisfies both the distance principle and the direction principle. The algorithm is divided into two parts, including the optimum incision and the optimum angle of laparoscope entry, which makes the laparoscope provide a reasonable initial visual field. A set of parameters based on the actual situation is given to illustrate the algorithm flow in detail. The preoperative planning algorithm offers significant improvements in planning time and quality for robot-assisted laparoscopic surgery. The improved method which combines the preoperative planning algorithm with deep deterministic policy gradient algorithm is applied to laparoscope arm automatic positioning for the robot-assisted laparoscopic surgery. It takes a fixed-point position and lesion parameters as input, and outputs the optimum incision, the optimum angle and motor movements without kinematics. The proposed algorithm is verified through simulations with a virtual environment built by pyglet. The results validate the correctness, feasibility, and robustness of this approach.

[1]  L W Sun,et al.  Advanced da Vinci surgical system simulator for surgeon training and operation planning , 2007, The international journal of medical robotics + computer assisted surgery : MRCAS.

[2]  T. Ahlering,et al.  Anatomic guide for port placement for daVinci robotic radical prostatectomy. , 2004, Journal of endourology.

[3]  Andreas Zell,et al.  Inverse Recurrent Models - An Application Scenario for Many-Joint Robot Arm Control , 2016, ICANN.

[4]  A. Cuschieri,et al.  Influence of direction of view, target‐to‐endoscope distance and manipulation angle on endoscopic knot tying , 1997, The British journal of surgery.

[5]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[6]  M. Menon,et al.  Optimizing robotic renal surgery: the lateral camera port placement technique and current results. , 2008, Journal of endourology.

[7]  R. Bauernschmitt,et al.  Optimal port placement and enhanced guidance in robotically assisted cardiac surgery , 2007, Surgical Endoscopy.

[8]  Andrea Salonia,et al.  Simplifying patient positioning and port placement during robotic-assisted laparoscopic prostatectomy. , 2010, European urology.

[9]  M Hashizume,et al.  Preoperative planning system for surgical robotics setup with kinematics and haptics , 2005, The international journal of medical robotics + computer assisted surgery : MRCAS.

[10]  Abe Fingerhut,et al.  Trocar placement for laparoscopic abdominal procedures: a simple standardized method. , 2004, Journal of the American College of Surgeons.

[11]  A. Cuschieri,et al.  Optimal port locations for endoscopic intracorporeal knotting , 1997, Surgical Endoscop.

[12]  Abhishek Sarkar,et al.  A deep reinforcement learning approach for dynamically stable inverse kinematics of humanoid robots , 2017, 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[13]  Lan Wang,et al.  Research on preoperative positioning analysis of instrument arms for minimally invasive surgical robot , 2014, 2014 IEEE International Conference on Mechatronics and Automation.

[14]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[15]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[16]  Naoki Suzuki,et al.  Robotic surgery setup simulation with the integration of inverse-kinematics computation and medical imaging , 2006, Comput. Methods Programs Biomed..

[17]  Ole Jakob Elle,et al.  Computer-aided planning of trocar placement and robot settings in robot-assisted surgery , 2001, CARS.

[18]  Yoshua Bengio,et al.  Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach , 2011, ICML.

[19]  Loi Wah Sun,et al.  Port placement and pose selection of the da Vinci surgical system for collision-free intervention based on performance optimization , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  J. Mulier,et al.  CT analysis of the elastic deformation and elongation of the abdominal wall during colon inflation for virtual coloscopy: 3AP6-7 , 2008 .

[21]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[22]  Berthold Bäuml,et al.  Optimal setup of the DLR MiroSurge telerobotic system for minimally invasive surgery , 2011, 2011 IEEE International Conference on Robotics and Automation.

[23]  A. Cuschieri,et al.  Influence of the optical axis-to-target view angle on endoscopic task performance , 1999, Surgical Endoscopy.

[24]  Masahiro Oda,et al.  Evaluation of deformation accuracy of a virtual pneumoperitoneum method based on clinical trials for patient-specific laparoscopic surgery simulator , 2012, Medical Imaging.

[25]  Guy Lever,et al.  Deterministic Policy Gradient Algorithms , 2014, ICML.

[26]  Sergey Levine,et al.  Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[27]  Stephen James,et al.  3D Simulation for Robot Arm Control with Deep Q-Learning , 2016, ArXiv.

[28]  Stefan Wermter,et al.  Neural End-to-End Learning of Reach for Grasp Ability with a 6-DoF Robot Arm , 2018 .

[29]  Terry M. Peters,et al.  Preoperative planning of robotics-assisted minimally invasive coronary artery bypass grafting , 2010, 2010 IEEE International Conference on Robotics and Automation.

[30]  Ma Ruq.  Preoperative Positioning Analysis of the Celiac Minimally Invasive Surgery Robotic System Based on an Improved Gradient Projection Algorithm , 2014 .