A knowledge-based framework for task automation in surgery

Robotic surgery has significantly improved the quality of surgical procedures. In the past, researches have been focused on automating simple surgical actions. However, there exists no scalable framework for automation in surgery. In this paper, we present a knowledge-based modular framework for the automation of articulated surgical tasks, for example, with multiple coordinated actions. The framework is consisted of ontology, providing entities for surgical automation and rules for task planning, and “dynamic movement primitives” as adaptive motion planner as to replicate the dexterity of surgeons. To validate our framework, we chose a paradigmatic scenario of a peg-and-ring task, a standard training exercise for novice surgeons which presents many challenges of real surgery, e.g. grasping and transferring. Experiments show the validity of the framework and its adaptability to faulty events. The modular architecture is expected to generalize to different tasks and platforms.

[1]  Stefan Schaal,et al.  Towards Associative Skill Memories , 2012, 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012).

[2]  Stefan Schaal,et al.  Biologically-inspired dynamical systems for movement generation: Automatic real-time goal adaptation and obstacle avoidance , 2009, 2009 IEEE International Conference on Robotics and Automation.

[3]  Nicola Guarino,et al.  Sweetening Ontologies with DOLCE , 2002, EKAW.

[4]  Kenneth Y. Goldberg,et al.  Automating multi-throw multilateral surgical suturing with a mechanical needle guide and sequential convex optimization , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[5]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[6]  Emilio Miguelanez,et al.  An IEEE standard Ontology for Robotics and Automation , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[7]  Maria Fox,et al.  PDDL2.1: An Extension to PDDL for Expressing Temporal Planning Domains , 2003, J. Artif. Intell. Res..

[8]  Thomas R. Gruber,et al.  A Translation Approach to Portable Ontologies , 1993 .

[9]  Riccardo Muradore,et al.  Development of a Cognitive Robotic System for Simple Surgical Tasks , 2015 .

[10]  Edwin Olson,et al.  Multipolicy decision-making for autonomous driving via changepoint-based behavior prediction: Theory and experiment , 2017, Auton. Robots.

[11]  Peter Stone,et al.  Task Planning in Robotics: an Empirical Comparison of PDDL-based and ASP-based Systems. , 2018 .

[12]  Yarden Katz,et al.  Pellet: A practical OWL-DL reasoner , 2007, J. Web Semant..

[13]  M. G. Mohanan,et al.  A survey of robotic motion planning in dynamic environments , 2018, Robotics Auton. Syst..

[14]  Jerzy W. Rozenblit,et al.  Modeling of a transfer task in computer assisted surgical training , 2016, SpringSim.

[15]  Tamim Asfour,et al.  Action sequence reproduction based on automatic segmentation and Object-Action Complexes , 2013, 2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids).

[16]  Paolo Fiorini,et al.  Dynamic Movement Primitives: Volumetric Obstacle Avoidance , 2019, 2019 19th International Conference on Advanced Robotics (ICAR).

[17]  Adam Pease,et al.  The Suggested Upper Merged Ontology: A Large Ontology for the Semantic Web and its Applic ations , 2002 .

[18]  Jun Morimoto,et al.  Orientation in Cartesian space dynamic movement primitives , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Guang-Zhong Yang,et al.  Hubot: A three state Human-Robot collaborative framework for bimanual surgical tasks based on learned models , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[20]  Barry Smith,et al.  SNAP and SPAN: Towards Dynamic Spatial Ontology , 2004, Spatial Cogn. Comput..

[21]  Nathaniel J Soper,et al.  The fundamentals of laparoscopic surgery: its time has come. , 2008, Bulletin of the American College of Surgeons.

[22]  Sergey Levine,et al.  End-to-End Training of Deep Visuomotor Policies , 2015, J. Mach. Learn. Res..

[23]  Danail Stoyanov,et al.  Surgical robotics beyond enhanced dexterity instrumentation: a survey of machine learning techniques and their role in intelligent and autonomous surgical actions , 2016, International Journal of Computer Assisted Radiology and Surgery.

[24]  Keno März,et al.  Toward a standard ontology of surgical process models , 2018, International Journal of Computer Assisted Radiology and Surgery.

[25]  Moritz Tenorth,et al.  KnowRob: A knowledge processing infrastructure for cognition-enabled robots , 2013, Int. J. Robotics Res..