A System for Robotic Heart Surgery that Learns to Tie Knots Using Recurrent Neural Networks

Tying suture knots is a time-consuming task performed frequently during minimally invasive surgery (MIS). Automating this task could greatly reduce total surgery time for patients. Current solutions to this problem replay manually programmed trajectories, but a more general and robust approach is to use supervised machine learning to smooth surgeon-given training trajectories and generalize from them. Since knot tying generally requires a controller with internal memory to distinguish between identical inputs that require different actions at different points along a trajectory, it would be impossible to teach the system using traditional feedforward neural nets or support vector machines. Instead we exploit more powerful, recurrent neural networks (RNNs) with adaptive internal states. Results obtained using long short-term memory RNNs trained by the recent Evolino algorithm show that this approach can significantly increase the efficiency of suture knot tying in MIS over preprogrammed control.

[1]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[2]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[3]  Eduardo Sontag,et al.  Turing computability with neural nets , 1991 .

[4]  Vladimir Vapnik,et al.  The Nature of Statistical Learning , 1995 .

[5]  A. García-Ruiz,et al.  Robotic surgical instruments for dexterity enhancement in thoracoscopic coronary artery bypass graft. , 1997, Journal of laparoendoscopic & advanced surgical techniques. Part A.

[6]  Didier Guériot,et al.  RBF neural network, basis functions and genetic algorithm , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[7]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[8]  A. García-Ruiz,et al.  Manual vs robotically assisted laparoscopic surgery in the performance of basic manipulation and suturing tasks. , 1998, Archives of surgery.

[9]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[10]  Risto Miikkulainen,et al.  Solving Non-Markovian Control Tasks with Neuro-Evolution , 1999, IJCAI.

[11]  Jürgen Schmidhuber,et al.  Learning to Forget: Continual Prediction with LSTM , 2000, Neural Computation.

[12]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[13]  John Kenneth Salisbury,et al.  The Intuitive/sup TM/ telesurgery system: overview and application , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[14]  Yoshua Bengio,et al.  Gradient Flow in Recurrent Nets: the Difficulty of Learning Long-Term Dependencies , 2001 .

[15]  Jürgen Schmidhuber,et al.  LSTM recurrent networks learn simple context-free and context-sensitive languages , 2001, IEEE Trans. Neural Networks.

[16]  Hyosig Kang,et al.  ROBOTIC ASSISTED SUTURING IN MINIMALLY INVASIVE SURGERY , 2002 .

[17]  Risto Miikkulainen,et al.  Active Guidance for a Finless Rocket Using Neuroevolution , 2003, GECCO.

[18]  Risto Miikkulainen,et al.  Robust non-linear control through neuroevolution , 2003 .

[19]  Alois Knoll,et al.  Endo[PA]R: An Open Evaluation System for Minimally Invasive Robotic Surgery , 2004 .

[20]  Risto Miikkulainen,et al.  Efficient Reinforcement Learning through Symbiotic Evolution , 2004 .

[21]  Jürgen Schmidhuber,et al.  Evolino: Hybrid Neuroevolution/Optimal Linear Search for Sequence Learning , 2005, IJCAI.

[22]  Jürgen Schmidhuber,et al.  Modeling systems with internal state using evolino , 2005, GECCO '05.

[23]  Jürgen Schmidhuber,et al.  A System for Robotic Heart Surgery that Learns to Tie Knots Using Recurrent Neural Networks , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[24]  Alois Knoll,et al.  Adaptive Control for Human-Robot Skilltransfer: Trajectory Planning Based on Fluid Dynamics , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[25]  Jürgen Schmidhuber,et al.  Training Recurrent Networks by Evolino , 2007, Neural Computation.