Optimization of virtual and real registration technology based on augmented reality in a surgical navigation system

BackgroundThe traditional navigation interface was intended only for two-dimensional observation by doctors; thus, this interface does not display the total spatial information for the lesion area. Surgical navigation systems have become essential tools that enable for doctors to accurately and safely perform complex operations. The image navigation interface is separated from the operating area, and the doctor needs to switch the field of vision between the screen and the patient’s lesion area. In this paper, augmented reality (AR) technology was applied to spinal surgery to provide more intuitive information to surgeons. The accuracy of virtual and real registration was improved via research on AR technology. During the operation, the doctor could observe the AR image and the true shape of the internal spine through the skin.MethodsTo improve the accuracy of virtual and real registration, a virtual and real registration technique based on an improved identification method and robot-assisted method was proposed. The experimental method was optimized by using the improved identification method. X-ray images were used to verify the effectiveness of the puncture performed by the robot.ResultsThe final experimental results show that the average accuracy of the virtual and real registration based on the general identification method was 9.73 ± 0.46 mm (range 8.90–10.23 mm). The average accuracy of the virtual and real registration based on the improved identification method was 3.54 ± 0.13 mm (range 3.36–3.73 mm). Compared with the virtual and real registration based on the general identification method, the accuracy was improved by approximately 65%. The highest accuracy of the virtual and real registration based on the robot-assisted method was 2.39 mm. The accuracy was improved by approximately 28.5% based on the improved identification method.ConclusionThe experimental results show that the two optimized methods are highly very effective. The proposed AR navigation system has high accuracy and stability. This system may have value in future spinal surgeries.

[1]  W. Sensakovic,et al.  CT Radiation Dose Reduction in Robot-assisted Pediatric Spinal Surgery , 2017, Spine.

[2]  Philip Pratt,et al.  Transoral Robotic Surgery: Image Guidance and Augmented Reality , 2018, ORL.

[3]  Ramin Javan,et al.  Head-mounted display augmented reality to guide pedicle screw placement utilizing computed tomography , 2018, International Journal of Computer Assisted Radiology and Surgery.

[4]  Samuel K. Cho,et al.  Navigation and Robotics in Spinal Surgery: Where Are We Now? , 2017, Neurosurgery.

[5]  G. Malham,et al.  What should my hospital buy next?-Guidelines for the acquisition and application of imaging, navigation, and robotics for spine surgery. , 2019, Journal of spine surgery.

[6]  Dimitris Mavrikios,et al.  A virtual and augmented reality approach to collaborative product design and demonstration , 2008, 2008 IEEE International Technology Management Conference (ICE).

[7]  S. Sadrameli,et al.  The use of robotics in minimally invasive spine surgery. , 2019, Journal of spine surgery.

[8]  Ho-Joong Kim,et al.  A prospective, randomized, controlled trial of robot‐assisted vs freehand pedicle screw fixation in spine surgery , 2017, The international journal of medical robotics + computer assisted surgery : MRCAS.

[9]  Kai-Che Liu,et al.  Real-time advanced spinal surgery via visible patient model and augmented reality system , 2014, Comput. Methods Programs Biomed..

[10]  B. Jaramaz,et al.  Computer Assisted Orthopaedic Surgery: Image Guided and Robotic Assistive Technologies , 1998, Clinical orthopaedics and related research.

[11]  T. Zhuang,et al.  [The method and development of computer-assisted surgery]. , 1998, Sheng wu yi xue gong cheng xue za zhi = Journal of biomedical engineering = Shengwu yixue gongchengxue zazhi.

[12]  Christopher Nimsky,et al.  Implementation of augmented reality support in spine surgery , 2019, European Spine Journal.

[13]  Eleonora Bottani,et al.  Augmented reality technology in the manufacturing industry: A review of the last decade , 2019, IISE Trans..

[14]  N. Navab,et al.  Advanced Medical Displays: A Literature Review of Augmented Reality , 2008, Journal of Display Technology.

[15]  Simon Weidert,et al.  A comparative analysis of intensity-based 2D–3D registration for intraoperative use in pedicle screw insertion surgeries , 2019, International Journal of Computer Assisted Radiology and Surgery.

[16]  M. Citardi,et al.  Image‐guided surgery: Fundamentals and clinical applications in otolaryngology Robert Labadie, J. Michael Fitzpatrick Plural Publishing, San Diego CA, 2016, USD 149.95, 215 pages , 2017 .

[17]  Gang Chai,et al.  [Registration technology for mandibular angle osteotomy based on augmented reality]. , 2010, Shanghai kou qiang yi xue = Shanghai journal of stomatology.

[18]  K. Cleary,et al.  Image-guided interventions: technology review and clinical applications. , 2010, Annual review of biomedical engineering.

[19]  Thomas J. Vogl,et al.  Robot-assisted percutaneous placement of K-wires during minimally invasive interventions of the spine , 2018, Minimally invasive therapy & allied technologies : MITAT : official journal of the Society for Minimally Invasive Therapy.

[20]  A. Darzi,et al.  Robot-assisted and fluoroscopy-guided pedicle screw placement: a systematic review , 2014, European Spine Journal.

[21]  Tianmiao Wang,et al.  A Review of Point Feature Based Medical Image Registration , 2018, Chinese Journal of Mechanical Engineering.

[22]  P. Dasgupta,et al.  Systematic review of augmented reality in urological interventions: the evidences of an impact on surgical outcomes are yet to come , 2019, World Journal of Urology.

[23]  Nico Verdonschot,et al.  Feasibility of A-mode ultrasound based intraoperative registration in computer-aided orthopedic surgery: A simulation and experimental study , 2018, PloS one.

[24]  Ichiro Sakuma,et al.  Real-time computer-generated integral imaging and 3D image calibration for augmented reality surgical navigation , 2015, Comput. Medical Imaging Graph..

[26]  Sait Naderi,et al.  Robotic systems in spine surgery. , 2014, Turkish neurosurgery.

[27]  Alexander Brawanski,et al.  Evaluation of robot-guided minimally invasive implantation of 2067 pedicle screws. , 2017, Neurosurgical focus.

[28]  Russell H. Taylor,et al.  Computer-Integrated Surgery: Technology and Clinical Applications , 1995 .

[29]  Hanwu He,et al.  Improving Registration of Augmented Reality by Incorporating DCNNS into Visual SLAM , 2018, Int. J. Pattern Recognit. Artif. Intell..

[30]  Florian Roser,et al.  Spinal robotics: current applications and future perspectives. , 2013, Neurosurgery.

[31]  V. Costalat,et al.  Robot-assisted spine surgery: feasibility study through a prospective case-matched analysis , 2016, European Spine Journal.

[32]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[33]  B. Fiani,et al.  Impact of robot-assisted spine surgery on health care quality and neurosurgical economics: A systemic review , 2018, Neurosurgical Review.

[34]  M. Rentschler,et al.  A new 3-dimensional method for measuring precision in surgical navigation and methods to optimize navigation accuracy , 2016, European Spine Journal.

[35]  A. Vaccaro,et al.  Robotic Guidance in Minimally Invasive Spine Surgery: a Review of Recent Literature and Commentary on a Developing Technology , 2019, Current Reviews in Musculoskeletal Medicine.

[36]  Leonid L. Chepelev,et al.  Applying Modern Virtual and Augmented Reality Technologies to Medical Images and Models , 2018, Journal of Digital Imaging.

[37]  Essam A. Rashed,et al.  An interactive augmented reality imaging system for minimally invasive orthopedic surgery , 2017, 2017 2nd International Conference on Knowledge Engineering and Applications (ICKEA).

[38]  Fang Chen,et al.  Augmented reality surgical navigation with ultrasound-assisted registration for pedicle screw placement: a pilot study , 2017, International Journal of Computer Assisted Radiology and Surgery.

[39]  Hilde van der Togt,et al.  Publisher's Note , 2003, J. Netw. Comput. Appl..

[40]  Luc Soler,et al.  Automatic localization of endoscope in intraoperative CT image: A simple approach to augmented reality guidance in laparoscopic surgery , 2016, Medical Image Anal..

[41]  Xiao-Guang Han,et al.  Guideline for Thoracolumbar Pedicle Screw Placement Assisted by Orthopaedic Surgical Robot , 2019, Orthopaedic surgery.

[42]  Victor X D Yang,et al.  Spinal intraoperative three-dimensional navigation: correlation between clinical and absolute engineering accuracy. , 2017, The spine journal : official journal of the North American Spine Society.

[43]  G Voss,et al.  ICAPS an integrative computer-assisted planning system for pedicle screw insertion. , 2001, Studies in health technology and informatics.

[44]  Michael Y. Wang,et al.  Workflow Caveats in Augmented Reality-Assisted Pedicle Instrumentation: Cadaver Lab. , 2019, World neurosurgery.

[45]  Enhai Liu,et al.  A single-image linear calibration method for camera , 2018 .

[46]  Martin Møller Jensen,et al.  Stereoscopic augmented reality system for supervised training on minimal invasive surgery robots , 2014, VRIC.

[47]  Jeffrey H Siewerdsen,et al.  Known-component 3D image reconstruction for improved intraoperative imaging in spine surgery: A clinical pilot study. , 2019, Medical physics.

[48]  Doniel Drazin,et al.  Robotics and the spine: a review of current and ongoing applications. , 2014, Neurosurgical focus.

[49]  Manoranjan Paul,et al.  A novel augmented reality (AR) scheme for knee replacement surgery by considering cutting error accuracy , 2019, The international journal of medical robotics + computer assisted surgery : MRCAS.

[50]  V. Rohde,et al.  Perioperative course and accuracy of screw positioning in conventional, open robotic-guided and percutaneous robotic-guided, pedicle screw placement , 2011, European Spine Journal.

[51]  N. Navab,et al.  Advanced Display and Visualization Concepts for Image Guided Surgery , 2008, Journal of Display Technology.

[52]  Liangcai Cao,et al.  Progress in virtual reality and augmented reality based on holographic display. , 2018, Applied optics.

[53]  Bo Zhang,et al.  Rapid regeneration offsets losses from warming-induced tree mortality in an aspen-dominated broad-leaved forest in northern China , 2018, PloS one.

[54]  He Liu,et al.  Augmented Reality Based Navigation for Computer Assisted Hip Resurfacing: A Proof of Concept Study , 2018, Annals of Biomedical Engineering.

[55]  Veit Rohde,et al.  Robotic versus fluoroscopy-guided pedicle screw insertion for metastatic spinal disease: a matched-cohort comparison. , 2017, Neurosurgical focus.

[56]  Hyunseok Choi,et al.  Perspective pinhole model with planar source for augmented reality surgical navigation based on C-arm imaging , 2018, International Journal of Computer Assisted Radiology and Surgery.

[57]  Hui Liu,et al.  Optimization Design and Performance Analysis of Vehicle Powertrain Mounting System , 2018, Chinese Journal of Mechanical Engineering.

[58]  Gustav Burström,et al.  A Novel Augmented-Reality-Based Surgical Navigation System for Spine Surgery in a Hybrid Operating Room: Design, Workflow, and Clinical Applications. , 2020, Operative neurosurgery.

[59]  L. Varich,et al.  Accuracy of robot-assisted pedicle screw placement for adolescent idiopathic scoliosis in the pediatric population , 2016, Journal of Robotic Surgery.

[60]  Sven R. Kantelhardt,et al.  Evaluation of surgical strategy of conventional vs. percutaneous robot-assisted spinal trans-pedicular instrumentation in spondylodiscitis , 2017, Journal of Robotic Surgery.

[61]  Manabu Ito,et al.  A novel 3D guidance system using augmented reality for percutaneous vertebroplasty: technical note. , 2013, Journal of neurosurgery. Spine.

[62]  Inverted C-arm Orientation During Simulated Hip Arthroscopic Surgery , 2018, Orthopaedic journal of sports medicine.

[63]  A. Adili Robot-Assisted Orthopedic Surgery , 2004, Seminars in laparoscopic surgery.

[64]  T. Jahng,et al.  Minimally Invasive Robotic Versus Open Fluoroscopic-guided Spinal Instrumented Fusions: A Randomized Controlled Trial. , 2017, Spine.

[65]  Florentin Liebmann,et al.  Augmented Reality Navigation for Spinal Pedicle Screw Instrumentation using Intraoperative 3D Imaging. , 2020, The spine journal : official journal of the North American Spine Society.

[66]  Jarod C Finlay,et al.  Deformable medical image registration of pleural cavity for photodynamic therapy by using finite-element based method , 2016, SPIE BiOS.

[67]  Georgios Sakas,et al.  Computer-aided surgery based on auto-stereoscopic augmented reality , 2004, Proceedings. Eighth International Conference on Information Visualisation, 2004. IV 2004..

[68]  Shaofang Lu,et al.  Virtual-Real Registration of Augmented Reality Technology Used in the Cerebral Surgery Lesion Localization , 2015, 2015 Fifth International Conference on Instrumentation and Measurement, Computer, Communication and Control (IMCCC).

[69]  Dieter Schmalstieg,et al.  HTC Vive MeVisLab integration via OpenVR for medical applications , 2017, PloS one.