Autonomous task planning and situation awareness in robotic surgery*

The use of robots in minimally invasive surgery has improved the quality of standard surgical procedures. So far, only the automation of simple surgical actions has been investigated by researchers, while the execution of structured tasks requiring reasoning on the environment and the choice among multiple actions is still managed by human surgeons. In this paper, we propose a framework to implement surgical task automation. The framework consists of a task-level reasoning module based on answer set programming, a low-level motion planning module based on dynamic movement primitives, and a situation awareness module. The logic-based reasoning module generates explainable plans and is able to recover from failure conditions, which are identified and explained by the situation awareness module interfacing to a human supervisor, for enhanced safety. Dynamic Movement Primitives allow to replicate the dexterity of surgeons and to adapt to obstacles and changes in the environment. The framework is validated on different versions of the standard surgical training peg-and-ring task.

[1]  P. Brett,et al.  An autonomous surgical robot for drilling a cochleostomy: preliminary porcine trial , 2008, Clinical otolaryngology : official journal of ENT-UK ; official journal of Netherlands Society for Oto-Rhino-Laryngology & Cervico-Facial Surgery.

[2]  Esra Erdem,et al.  Developing and Maintaining an Ontology for Rehabilitation Robotics , 2012, KEOD.

[3]  Cesare Tinelli,et al.  Solving SAT and SAT Modulo Theories: From an abstract Davis--Putnam--Logemann--Loveland procedure to DPLL(T) , 2006, JACM.

[4]  Wyatt S. Newman,et al.  A novel vision guided knot-tying method for autonomous robotic surgery , 2014, 2014 IEEE International Conference on Automation Science and Engineering (CASE).

[5]  Silvia Coradeschi,et al.  Using Knowledge Representation for Perceptual Anchoring in a Robotic System , 2008, Int. J. Artif. Intell. Tools.

[6]  M. Mack,et al.  Minimally invasive and robotic surgery. , 2001, JAMA.

[7]  K. M. Deliparaschos,et al.  Evolution of autonomous and semi‐autonomous robotic surgical systems: a review of the literature , 2011, The international journal of medical robotics + computer assisted surgery : MRCAS.

[8]  Stefan Schaal,et al.  Biologically-inspired dynamical systems for movement generation: Automatic real-time goal adaptation and obstacle avoidance , 2009, 2009 IEEE International Conference on Robotics and Automation.

[9]  Paolo Fiorini,et al.  A knowledge-based framework for task automation in surgery , 2019, 2019 19th International Conference on Advanced Robotics (ICAR).

[10]  Moritz Tenorth,et al.  KnowRob: A knowledge processing infrastructure for cognition-enabled robots , 2013, Int. J. Robotics Res..

[11]  Peter Kazanzides,et al.  Medical robotics—Regulatory, ethical, and legal considerations for increasing levels of autonomy , 2017, Science Robotics.

[12]  Paolo Fiorini,et al.  Dynamic Movement Primitives: Volumetric Obstacle Avoidance , 2019, 2019 19th International Conference on Advanced Robotics (ICAR).

[13]  F. Pirozzi,et al.  Advantages and limits of robot-assisted laparoscopic surgery: preliminary experience , 2004, Surgical Endoscopy And Other Interventional Techniques.

[14]  Danny De Schreye,et al.  Answer Set Planning , 1999 .

[15]  Kenneth Y. Goldberg,et al.  Automating multi-throw multilateral surgical suturing with a mechanical needle guide and sequential convex optimization , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[16]  Miroslaw Truszczynski,et al.  Preferences and Nonmonotonic Reasoning , 2008, AI Mag..

[17]  Riccardo Muradore,et al.  Development of a Cognitive Robotic System for Simple Surgical Tasks , 2015 .

[18]  Matteo Saveriano,et al.  Merging Position and orientation Motion Primitives , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[19]  Maria Fox,et al.  PDDL2.1: An Extension to PDDL for Expressing Temporal Planning Domains , 2003, J. Artif. Intell. Res..

[20]  Gerhard Friedrich,et al.  Industrial Applications of Answer Set Programming , 2018, KI - Künstliche Intelligenz.

[21]  Mohamed R Ali,et al.  Robotic cholecystectomy: learning curve, advantages, and limitations. , 2006, The Journal of surgical research.

[22]  Michael Gelfond,et al.  The USA-Advisor: A Case Study in Answer Set Planning , 2001, LPNMR.

[23]  Bernhard Nebel,et al.  Encoding Planning Problems in Nonmonotonic Logic Programs , 1997, ECP.

[24]  David B. Camarillo,et al.  Robotic technology in surgery: past, present, and future. , 2004, American journal of surgery.

[25]  Martin Gebser,et al.  Routing Driverless Transport Vehicles in Car Assembly with Answer Set Programming , 2018, Theory Pract. Log. Program..

[26]  Paolo Fiorini,et al.  DMP++: Overcoming Some Drawbacks of Dynamic Movement Primitives , 2019, Robotics Auton. Syst..

[27]  Brijen Thananjeyan,et al.  Applying Depth-Sensing to Automated Surgical Manipulation with a da Vinci Robot , 2020, 2020 International Symposium on Medical Robotics (ISMR).

[28]  Danail Stoyanov,et al.  Surgical robotics beyond enhanced dexterity instrumentation: a survey of machine learning techniques and their role in intelligent and autonomous surgical actions , 2016, International Journal of Computer Assisted Radiology and Surgery.

[29]  Heinz Wörn,et al.  Cognitive Robotics in Industrial Environments , 2012 .

[30]  Alain Colmerauer,et al.  An introduction to Prolog III , 1989, CACM.