Integration of Virtual Reality in the Control System of an Innovative Medical Robot for Single-Incision Laparoscopic Surgery

In recent years, there has been an expansion in the development of simulators that use virtual reality (VR) as a learning tool. In surgery where robots are used, VR serves as a revolutionary technology to help medical doctors train in using these robotic systems and accumulate knowledge without risk. This article presents a study in which VR is used to create a simulator designed for robotically assisted single-uniport surgery. The control of the surgical robotic system is achieved using voice commands for laparoscopic camera positioning and via a user interface developed using the Visual Studio program that connects a wristband equipped with sensors attached to the user’s hand for the manipulation of the active instruments. The software consists of the user interface and the VR application via the TCP/IP communication protocol. To study the evolution of the performance of this virtual system, 15 people were involved in the experimental evaluation of the VR simulator built for the robotic surgical system, having to complete a medically relevant task. The experimental data validated the initial solution, which will be further developed.

[1]  K. Bliznakova,et al.  Assessment of students' satisfaction with virtual robotic surgery training , 2023, Heliyon.

[2]  B. Shirinzadeh,et al.  A Transparent Teleoperated Robotic Surgical System with Predictive Haptic Feedback and Force Modelling , 2022, Sensors.

[3]  B. Gherman,et al.  Design and Experimental Setup of a Robotic Medical Instrument for Brachytherapy in Non-Resectable Liver Tumors , 2022, Cancers.

[4]  A. Okamura,et al.  A Modular 3-Degree-of-Freedom Force Sensor for Robot-assisted Minimally Invasive Surgery Research , 2022, ArXiv.

[5]  M. Korayem,et al.  Design, manufacture, and control of a laparoscopic robot via Leap Motion sensors , 2022, Measurement.

[6]  Moharam Habibnejad Korayem,et al.  Detecting Hand's Tremor Using Leap Motion Controller in Guiding Surgical Robot Arms and Laparoscopic Scissors , 2022, SSRN Electronic Journal.

[7]  B. Shirinzadeh,et al.  A Force-Feedback Methodology for Teleoperated Suturing Task in Robotic-Assisted Minimally Invasive Surgery , 2022, Sensors.

[8]  D. Chablat,et al.  A review of cuspidal serial and parallel manipulators , 2022, Journal of Mechanisms and Robotics.

[9]  B. Gherman,et al.  Application Oriented Modelling and Simulation of an Innovative Parallel Robot for Single Incision Laparoscopic Surgery , 2022, Volume 7: 46th Mechanisms and Robotics Conference (MR).

[10]  A. de Giorgio,et al.  Extended Reality in Neurosurgical Education: A Systematic Review , 2022, Sensors.

[11]  A. Iordan,et al.  Control of a Drone in Virtual Reality Using MEMS Sensor Technology and Machine Learning , 2022, Micromachines.

[12]  Anuradha Ranasinghe,et al.  A Novel Untethered Hand Wearable with Fine-Grained Cutaneous Haptic Feedback , 2022, Sensors.

[13]  G. Umana,et al.  Virtual Reality in Neurosurgery: Beyond Neurosurgical Planning , 2022, International journal of environmental research and public health.

[14]  Valena J. Wright,et al.  Developing basic robotic skills using virtual reality simulation and automated assessment tools: a multidisciplinary robotic virtual reality-based curriculum using the Da Vinci Skills Simulator and tracking progress with the Intuitive Learning platform , 2022, Journal of Robotic Surgery.

[15]  P. Dasgupta,et al.  Effect of Simulation-based Training on Surgical Proficiency and Patient Outcomes: A Randomised Controlled Clinical and Educational Trial. , 2021, European urology.

[16]  Daniel Leidner,et al.  A Digital Twin Approach for Contextual Assistance for Surgeons During Surgical Robotics Training , 2021, Frontiers in Robotics and AI.

[17]  J. Kay,et al.  Immersive Virtual Reality for Surgical Training: A Systematic Review. , 2021, The Journal of surgical research.

[18]  M.H. Korayem,et al.  Controlling surgical robot arm using leap motion controller with Kalman filter , 2021 .

[19]  Catalin Dumitrescu,et al.  Fuzzy Logic for Intelligent Control System Using Soft Computing Applications , 2021, Sensors.

[20]  A. Farrokh,et al.  Evaluation of Laparoscopy Virtual Reality Training on the Improvement of Trainees’ Surgical Skills , 2021, Medicina.

[21]  Adrian Pisla,et al.  Development of a Virtual Reality Simulator for an Intelligent Robotic System Used in Ankle Rehabilitation , 2021, Sensors.

[22]  Nassir Navab,et al.  On the effectiveness of virtual reality-based training for surgical robot setup , 2020, Comput. methods Biomech. Biomed. Eng. Imaging Vis..

[23]  Joseph Hosanna Raj Isaac,et al.  Novel virtual reality based training system for fine motor skills: Towards developing a robotic surgery training system , 2020, The international journal of medical robotics + computer assisted surgery : MRCAS.

[24]  J. Abela,et al.  Virtual reality training compared with apprenticeship training in laparoscopic surgery: a meta-analysis. , 2020, Annals of the Royal College of Surgeons of England.

[25]  L. Gesualdo,et al.  Innovative Educational Pathways in Spine Surgery: Advanced Virtual Reality-Based Training. , 2020, World neurosurgery.

[26]  Carole S Franklin,et al.  Collaborative robotics: New era of human-robot cooperation in the workplace. , 2020, Journal of safety research.

[27]  Ganesh R. Naik,et al.  Laparoscopic Robotic Surgery: Current Perspective and Future Directions , 2020, Robotics.

[28]  R. Satava,et al.  Demonstrating the effectiveness of the fundamentals of robotic surgery (FRS) curriculum on the RobotiX Mentor Virtual Reality Simulation Platform , 2020, Journal of Robotic Surgery.

[29]  Doina Pisla,et al.  Development of a Virtual Reality Simulator for a Lower Limb Rehabilitation Robot , 2020, 2020 IEEE International Conference on Automation, Quality and Testing, Robotics (AQTR).

[30]  G. Dubernard,et al.  Virtual reality simulation to enhance laparoscopic salpingectomy skills. , 2020, Journal of gynecology obstetrics and human reproduction.

[31]  K. Kawashima,et al.  Robots in laparoscopic surgery: current and future status , 2019, BMC biomedical engineering.

[32]  Giuseppe Carbone,et al.  Risk-Based Assessment Engineering of a Parallel Robot Used in Post-Stroke Upper Limb Rehabilitation , 2019, Sustainability.

[33]  Alberto Borboni,et al.  Finite element method stiffness analysis of a novel telemanipulator for minimally invasive surgery , 2019, Simul..

[34]  A E Iordan,et al.  Optimal solution of the Guarini puzzle extension using tripartite graphs , 2019, IOP Conference Series: Materials Science and Engineering.

[35]  Jie Cao,et al.  Single-incision laparoscopic versus conventional laparoscopic right colectomy: A systematic review and meta-analysis. , 2018 .

[36]  Anany Levitin,et al.  Algorithmic Puzzles: History, Taxonomies, and Applications in Human Problem Solving , 2017, J. Probl. Solving.

[37]  P. Breedveld,et al.  A state of the art review and categorization of multi-branched instruments for NOTES and SILS , 2015, Surgical Endoscopy.

[38]  Caius Panoiu,et al.  Artificial neural networks in predicting current in electric arc furnaces , 2014 .

[39]  P. Morel,et al.  History of robotic surgery: from AESOP® and ZEUS® to da Vinci®. , 2011, Journal of visceral surgery.

[40]  F Graur,et al.  PARASURG hybrid parallel robot for minimally invasive surgery. , 2011, Chirurgia.

[41]  B. Gherman,et al.  Singularities and workspace analysis for a parallel robot for minimally invasive surgery , 2010, 2010 IEEE International Conference on Automation, Quality and Testing, Robotics (AQTR).

[42]  Anca Iordan,et al.  Development of interactive software for teaching three-dimensional analytic geometry , 2009 .

[43]  M. Stifelman,et al.  Robot assisted partial nephrectomy versus laparoscopic partial nephrectomy for renal tumors: a multi-institutional analysis of perioperative outcomes. , 2009, The Journal of urology.

[44]  Doina Pisla,et al.  Kinematics and design of two variants of a reconfigurable parallel robot , 2009, 2009 ASME/IFToMM International Conference on Reconfigurable Mechanisms and Robots.

[45]  G. Carbone,et al.  Rehabilitation System with Integrated Visual Stimulation , 2020 .

[46]  Florin Covaciu,et al.  IMPROVING DESIGN OF A TRIANGLE GEOMETRY COMPUTER APPLICATION USING A CREATIONAL PATTERN , 2020 .

[47]  Nicolae Plitea,et al.  Development of a Voice Controlled Surgical Robot , 2010 .

[48]  Arne Burisch,et al.  Innovative development of surgical parallel robots , 2007 .