Cognitive Robotic Architecture for Semi-Autonomous Execution of Manipulation Tasks in a Surgical Environment

The development of robotic systems with a certain level of autonomy to be used in critical scenarios, such as an operating room, necessarily requires a seamless integration of multiple state-of-the-art technologies. In this paper we propose a cognitive robotic architecture that is able to help an operator accomplish a specific task. The architecture integrates an action recognition module to understand the scene, a supervisory control to make decisions, and a model predictive control to plan collision-free trajectory for the robotic arm taking into account obstacles and model uncertainty. The proposed approach has been validated on a simplified scenario involving only a da VinciO surgical robot and a novel manipulator holding standard laparoscopic tools.

[1]  Riccardo Muradore,et al.  Enhancing Surgical Process Modeling for Artificial Intelligence development in robotics: the SARAS case study for Minimally Invasive Procedures , 2019, 2019 13th International Symposium on Medical Information and Communication Technology (ISMICT).

[2]  Riccardo Muradore,et al.  An energy-shared two-layer approach for multi-master-multi-slave bilateral teleoperation systems , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[3]  Ratna Babu Chinnam,et al.  Soft Boundary Approach for Unsupervised Gesture Segmentation in Robotic-Assisted Surgery , 2017, IEEE Robotics and Automation Letters.

[4]  Andru Putra Twinanda,et al.  EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos , 2016, IEEE Transactions on Medical Imaging.

[5]  Rafael Muñoz-Salinas,et al.  Speeded up detection of squared fiducial markers , 2018, Image Vis. Comput..

[6]  Vladimir J. Lumelsky,et al.  On Fast Computation of Distance Between Line Segments , 1985, Information Processing Letters.

[7]  James W. Davis,et al.  The Representation and Recognition of Action Using Temporal Templates , 1997, CVPR 1997.

[8]  Henry C. Lin,et al.  JHU-ISI Gesture and Skill Assessment Working Set ( JIGSAWS ) : A Surgical Activity Dataset for Human Motion Modeling , 2014 .

[9]  Hongmin Wu,et al.  Base frame calibration for multi-robot coordinated systems , 2015, 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[10]  Pierre Jannin,et al.  Knowledge transfer for surgical activity prediction , 2018, International Journal of Computer Assisted Radiology and Surgery.

[11]  Bruno Siciliano,et al.  Autonomy in surgical robots and its meaningful human control , 2019, Paladyn J. Behav. Robotics.

[12]  Cristian Secchi,et al.  Safety Barrier Functions for Human-Robot Interaction with Industrial Manipulators , 2019, 2019 18th European Control Conference (ECC).

[13]  Peter Kazanzides,et al.  Medical robotics—Regulatory, ethical, and legal considerations for increasing levels of autonomy , 2017, Science Robotics.

[14]  Riccardo Muradore,et al.  A Multirobots Teleoperated Platform for Artificial Intelligence Training Data Collection in Minimally Invasive Surgery , 2019, 2019 International Symposium on Medical Robotics (ISMR).

[15]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Gregory D. Hager,et al.  Segmental Spatiotemporal CNNs for Fine-Grained Action Segmentation , 2016, ECCV.

[17]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[18]  Gregory D. Hager,et al.  Transition state clustering: Unsupervised surgical trajectory segmentation for robot learning , 2017, ISRR.