Design of Navigation System for Liver Surgery Guided by Augmented Reality

The traditional surgical navigation system combining preoperative CT and intraoperative ultrasound is widely used in open hepatectomy, but the problem of this system is that there are some errors in terms of the precision of the time and the space during the surgery. In order to solve the above problems, this paper introduces augmented reality technology into the surgical navigation system. In order to accurately describe the biomechanical characteristics of the liver and let the navigation system perform more accurately, this paper uses Tetgen to perform tetrahedral partition on the triangle mesh data obtained after the three-dimensional CT was reconstructed, and then the surface mesh data and the internal tetrahedral data were obtained and they were used to describe a whole liver model. The surface triangle mesh data is used to render graphics and describe the surface topology change, and the internal tetrahedral data combined with mass-spring theory is used to simulate deformation. Subsequently, the ex vivo pig liver was used to experimentally verify the accuracy of gravity deformation of the liver model, and the results show that the error was mainly distributed between −2mm and −2.5mm. At the same time, this paper uses the NDI Polaris infrared tracking system to carry out precision experiments on the augmented reality module, and the measured error is 1.55± 0.29mm. Finally, various modules of the system are integrated to finish the experiment in which the ring-shaped lesions are cut from the ex vivo pig liver with the aid of augmented reality. The experimental error is 0± 1.26mm, and with the assistance of the general purpose graphics processing unit (GPGPU), the refresh rate is above 200FPS. The results prove that the liver surgery navigation system proposed in this paper is excellent in terms of real-time performance and accuracy, which can help doctors accurately locate the tumor during surgery and perform ideal resection.

[1]  Adrian Saftoiu,et al.  The role of real-time contrast-enhanced and real-time virtual sonography in the assessment of malignant liver lesions. , 2009, Journal of gastrointestinal and liver diseases : JGLD.

[2]  Yoshihiro Sakamoto,et al.  Initial experience of intraoperative three-dimensional navigation for liver resection using real-time virtual sonography. , 2014, Surgery.

[3]  C. Lawson Software for C1 Surface Interpolation , 1977 .

[4]  Stephane Cotin,et al.  Marker-Based Registration for Large Deformations - Application to Open Liver Surgery , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[5]  J. Strohbehn,et al.  A frameless stereotaxic integration of computerized tomographic imaging and the operating microscope. , 1986, Journal of neurosurgery.

[6]  Naoki Suzuki,et al.  Utility of augmented reality system in hepatobiliary surgery , 2013, Journal of hepato-biliary-pancreatic sciences.

[7]  Jiann-Der Lee,et al.  A Projection-based AR system to display brain angiography via Stereo Vision , 2018, 2018 IEEE 7th Global Conference on Consumer Electronics (GCCE).

[8]  Akio Saiura,et al.  Feasibility of Intraoperative Navigation for Liver Resection Using Real-time Virtual Sonography With Novel Automatic Registration System , 2018, World Journal of Surgery.

[9]  Guang-Zhong Yang,et al.  Comparative effectiveness and safety of image guidance systems in surgery: a preclinical randomised study , 2015, The Lancet.

[10]  Marco Nolden,et al.  Real-time image guidance in laparoscopic liver surgery: first clinical experience with a guidance system based on intraoperative CT imaging , 2013, Surgical Endoscopy.

[11]  Luca Viganò,et al.  Surgery of Colorectal Liver Metastases: Pushing the Limits , 2016, Liver Cancer.

[12]  Makoto Hashizume,et al.  Image-guided laparoscopic surgery in an open MRI operating theater , 2013, Surgical Endoscopy.

[13]  M. Makuuchi,et al.  Ultrasonically guided subsegmentectomy. , 1985, Surgery, gynecology & obstetrics.

[14]  Stéphane Cotin,et al.  Augmented Reality during Open Liver Surgery Using a Markerless Non-rigid Registration System , 2020, Journal of Gastrointestinal Surgery.

[15]  Gábor Székely,et al.  A New Approach to Cutting into Finite Element Models , 2001, MICCAI.

[16]  Michael I Miga,et al.  Evaluation of model-based deformation correction in image-guided liver surgery via tracked intraoperative ultrasound , 2016, Journal of medical imaging.

[17]  Pheng-Ann Heng,et al.  Augmented Reality-Based Personalized Virtual Operative Anatomy for Neurosurgical Guidance and Training , 2018, 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[18]  Lejing Wang,et al.  Camera‐augmented mobile C‐arm (CamC): A feasibility study of augmented reality imaging in the operating room , 2018, The international journal of medical robotics + computer assisted surgery : MRCAS.

[19]  Aiqun Zhang,et al.  Precision in Liver Surgery , 2013, Seminars in Liver Disease.

[20]  Luc Soler,et al.  Trans-thoracic minimally invasive liver resection guided by augmented reality. , 2015, Journal of the American College of Surgeons.

[21]  Monan Wang,et al.  A review of virtual cutting methods and technology in deformable objects , 2018, The international journal of medical robotics + computer assisted surgery : MRCAS.

[22]  Luc Soler,et al.  Augmented Reality Guidance for the Resection of Missing Colorectal Liver Metastases: An Initial Experience , 2016, World Journal of Surgery.

[23]  Han-Wen Nienhuys,et al.  A Surgery Simulation Supporting Cuts and Finite Element Deformation , 2001, MICCAI.

[24]  Kazuma Fujimoto,et al.  Radiofrequency Ablation with the Real-Time Virtual Sonography System for Treating Hepatocellular Carcinoma Difficult to Detect by Ultrasonography , 2006, Journal of clinical biochemistry and nutrition.

[25]  Ekkehard Euler,et al.  Visual Servoing for Intraoperative Positioning and Repositioning of Mobile C-arms , 2006, MICCAI.

[26]  Teppei Tsujita,et al.  AR Brain-Shift Display for Computer-Assisted Neurosurgery , 2019, 2019 58th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE).

[27]  Fabrizio Cutolo,et al.  Software Framework for Customized Augmented Reality Headsets in Medicine , 2020, IEEE Access.

[28]  Wei Qian,et al.  Registration Technology of Augmented Reality in Oral Medicine: A Review , 2019, IEEE Access.

[29]  H. Liao,et al.  Augmented reality technology for preoperative planning and intraoperative navigation during hepatobiliary surgery: A review of current methods. , 2018, Hepatobiliary & pancreatic diseases international : HBPD INT.

[30]  Simon Wildermuth,et al.  A survey of interactive mesh-cutting techniques and a new method for implementing generalized interactive mesh cutting using virtual tools , 2002, Comput. Animat. Virtual Worlds.

[31]  Nassir Navab,et al.  Camera Augmented Mobile C-Arm (CAMC): Calibration, Accuracy Study, and Clinical Applications , 2010, IEEE Transactions on Medical Imaging.

[32]  H Delingette,et al.  Virtual reality applied to hepatic surgery simulation: the next revolution. , 1998, Annals of surgery.

[33]  J. Marescaux,et al.  Augmented reality in laparoscopic surgical oncology. , 2011, Surgical oncology.

[34]  J. Marescaux,et al.  Towards cybernetic surgery: robotic and augmented reality-assisted liver segmentectomy , 2015, Langenbeck's Archives of Surgery.

[35]  D. Caleb Rucker,et al.  A Mechanics-Based Nonrigid Registration Method for Liver Surgery Using Sparse Intraoperative Data , 2014, IEEE Transactions on Medical Imaging.