Real-time 3D image reconstruction guidance in liver resection surgery.

BACKGROUND Minimally invasive surgery represents one of the main evolutions of surgical techniques. However, minimally invasive surgery adds difficulty that can be reduced through computer technology. METHODS From a patient's medical image [US, computed tomography (CT) or MRI], we have developed an Augmented Reality (AR) system that increases the surgeon's intraoperative vision by providing a virtual transparency of the patient. AR is based on two major processes: 3D modeling and visualization of anatomical or pathological structures appearing in the medical image, and the registration of this visualization onto the real patient. We have thus developed a new online service, named Visible Patient, providing efficient 3D modeling of patients. We have then developed several 3D visualization and surgical planning software tools to combine direct volume rendering and surface rendering. Finally, we have developed two registration techniques, one interactive and one automatic providing intraoperative augmented reality view. RESULTS From January 2009 to June 2013, 769 clinical cases have been modeled by the Visible Patient service. Moreover, three clinical validations have been realized demonstrating the accuracy of 3D models and their great benefit, potentially increasing surgical eligibility in liver surgery (20% of cases). From these 3D models, more than 50 interactive AR-assisted surgical procedures have been realized illustrating the potential clinical benefit of such assistance to gain safety, but also current limits that automatic augmented reality will overcome. CONCLUSIONS Virtual patient modeling should be mandatory for certain interventions that have now to be defined, such as liver surgery. Augmented reality is clearly the next step of the new surgical instrumentation but remains currently limited due to the complexity of organ deformations during surgery. Intraoperative medical imaging used in new generation of automated augmented reality should solve this issue thanks to the development of Hybrid OR.

[1]  C. Couinaud,et al.  Liver Anatomy: Portal (and Suprahepatic) or Biliary Segmentation , 2000, Digestive Surgery.

[2]  Ziv Yaniv,et al.  Needle-Based Interventions With the Image-Guided Surgery Toolkit (IGSTK): From Phantoms to Clinical Trials , 2010, IEEE Transactions on Biomedical Engineering.

[3]  Hans-Peter Meinzer,et al.  A Statistical Deformable Model for the Segmentation of Liver CT Volumes , 2007 .

[4]  T. Vogl,et al.  Navigation-Based Needle Puncture of a Cadaver Using a Hybrid Tracking Navigational System , 2006, Investigative radiology.

[5]  Johan Montagnat,et al.  Fully Automatic Anatomical, Pathological, and Functional Segmentation from CT Scans for Hepatic Surgery , 2000, Medical Imaging: Image Processing.

[6]  Luc Soler,et al.  An augmented reality system for liver thermal ablation: Design and evaluation on clinical cases , 2009, Medical Image Anal..

[7]  Nassir Navab,et al.  Magneto-Optical Tracking of Flexible Laparoscopic Ultrasound: Model-Based Online Detection and Correction of Magnetic Tracking Errors , 2009, IEEE Transactions on Medical Imaging.

[8]  A. Park,et al.  Live augmented reality: a new visualization method for laparoscopic surgery using continuous volumetric computed tomography , 2010, Surgical Endoscopy.

[9]  Thomas Lange,et al.  Automatic segmentation of the liver for preoperative planning of resections. , 2003, Studies in health technology and informatics.

[10]  Jong-Won Park,et al.  Automatic Segmentation Technique Without User Modification for 3D Visualization in Medical Images , 2004, CIS.

[11]  J Marescaux,et al.  A real-time predictive simulation of abdominal viscera positions during quiet free breathing. , 2010, Progress in biophysics and molecular biology.

[12]  Nassir Navab,et al.  Adaptive visualization for needle guidance in RF liver ablation: taking organ deformation into account , 2008, SPIE Medical Imaging.

[13]  Yo-Sung Ho,et al.  Automatic liver segmentation for volume measurement in CT Images , 2006, J. Vis. Commun. Image Represent..

[14]  H P Meinzer,et al.  The impact of 3-dimensional reconstructions on operation planning in liver surgery. , 2000, Archives of surgery.

[15]  Ken Masamune,et al.  Image overlay guidance for needle insertion in CT scanner , 2005, IEEE Transactions on Biomedical Engineering.

[16]  H. Bismuth,et al.  Surgical anatomy and anatomical surgery of the liver , 2005, World Journal of Surgery.

[17]  Gabor Fichtinger,et al.  MRI image overlay: Application to arthrography needle insertion , 2007, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.

[18]  Marc Jaeger,et al.  Advanced 3D image processing techniques for liver and hepatic tumor location and volumetry , 1999, Medical Imaging.

[19]  Jing Bai,et al.  Atlas based automatic identification of abdominal organs , 2005, SPIE Medical Imaging.

[20]  Frank Sauer,et al.  An augmented reality system for MR image-guided needle biopsy: initial results in a swine model. , 2006, Radiology.

[21]  László Ruskó,et al.  Automatic segmentation of the liver from multi- and single-phase contrast-enhanced CT images , 2009, Medical Image Anal..

[22]  E. Molmenti,et al.  Territorial belonging of the middle hepatic vein in living liver donor candidates evaluated by three‐dimensional computed tomographic reconstruction and virtual liver resection , 2009, The British journal of surgery.

[23]  Hyunjin Park,et al.  Construction of an abdominal probabilistic atlas and its application in segmentation , 2003, IEEE Transactions on Medical Imaging.

[24]  J.D. Furst,et al.  Single Organ Segmentation Filters for Multiple Organ Segmentation , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[25]  J. Marescaux,et al.  Augmented-reality-assisted laparoscopic adrenalectomy. , 2004, JAMA.

[26]  C Couinaud,et al.  Erreur dans le diagnostic topographique des lésions hépatiques , 2002 .

[27]  Luc Soler,et al.  Bulk modulus and volume variation measurement of the liver and the kidneys in vivo using abdominal kinetics during free breathing , 2010, Comput. Methods Programs Biomed..

[28]  Jacob D. Furst,et al.  Single click volumetric segmentation of abdominal organs in computed tomography images , 2006, SPIE Medical Imaging.

[29]  D Mutter,et al.  Recent advances in liver imaging , 2010, Expert review of gastroenterology & hepatology.

[30]  C Couinaud.  [Errors in the topographic diagnosis of liver diseases]. , 2002, Annales de chirurgie.

[31]  Luc Soler,et al.  PACS-based interface for 3D anatomical structure visualization and surgical planning , 2002, SPIE Medical Imaging.

[32]  Makoto Hashizume,et al.  Recovery of respiratory motion and deformation of the liver using laparoscopic freehand 3D ultrasound system , 2007, Medical Image Anal..

[33]  Isabelle Bloch,et al.  Computational modeling of thoracic and abdominal anatomy using spatial relationships for image segmentation , 2004, Real Time Imaging.