Surgical navigation by autostereoscopic image overlay of integral videography

This paper describes an autostereoscopic image overlay technique that is integrated into a surgical navigation system to superimpose a real three-dimensional (3-D) image onto the patient via a half-silvered mirror. The images are created by employing a modified version of integral videography (IV), which is an animated extension of integral photography. IV records and reproduces 3-D images using a microconvex lens array and flat display; it can display geometrically accurate 3-D autostereoscopic images and reproduce motion parallax without the need for special devices. The use of semitransparent display devices makes it appear that the 3-D image is inside the patient's body. This is the first report of applying an autostereoscopic display with an image overlay system in surgical navigation. Experiments demonstrated that the fast IV rendering technique and patient-image registration method produce an average registration accuracy of 1.13 mm. Experiments using a target in phantom agar showed that the system can guide a needle toward a target with an average error of 2.6 mm. Improvement in the quality of the IV display will make this system practical and its use will increase surgical accuracy and reduce invasiveness.

[1]  Haiying Liu,et al.  Investigation of intraoperative brain deformation using a 1.5-T interventional MR system: preliminary results , 1998, IEEE Transactions on Medical Imaging.

[2]  Jay B. West,et al.  Implementation, calibration and accuracy testing of an image-enhanced endoscopy system , 2002, IEEE Transactions on Medical Imaging.

[3]  Torsten Rohlfing,et al.  Augmented-reality visualization of brain structures with stereo and kinetic depth cues: system description and initial evaluation with head phantom , 2001, SPIE Medical Imaging.

[4]  G. Lippmann.  Epreuves reversibles donnant la sensation du relief , 1908 .

[5]  Yongmin Kim,et al.  Interactive 3D registration of ultrasound and magnetic resonance images based on a magnetic position sensor , 1999, IEEE Transactions on Information Technology in Biomedicine.

[6]  Takeo Kanade,et al.  An Image Overlay System for Medical Data Visualization , 1998, MICCAI.

[7]  Wolfgang Birkfellner,et al.  A head-mounted operating binocular for augmented reality visualization in medicine - design and initial evaluation , 2002, IEEE Transactions on Medical Imaging.

[8]  Frank Sauer,et al.  A Head-Mounted Display System for Augmented Reality Image Guidance: Towards Clinical Evaluation for iMRI-guided Neurosurgery , 2001, MICCAI.

[9]  Frank Langlotz,et al.  State-of-the-art in orthopaedic surgical navigation with a focus on medical image modalities , 2002, Comput. Animat. Virtual Worlds.

[10]  Ramesh Raskar,et al.  Augmented Reality Visualization for Laparoscopic Surgery , 1998, MICCAI.

[11]  Henry Fuchs,et al.  Augmented reality guidance for needle biopsies: An initial randomized, controlled trial in phantoms , 2002, Medical Image Anal..

[12]  K Masamune,et al.  A new safe laparoscopic manipulator system with a five-bar linkage mechanism and an optical zoom. , 1999, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.

[13]  David J. Hawkes,et al.  Design and evaluation of a system for microscope-assisted guided interventions (MAGI) , 1999, IEEE Transactions on Medical Imaging.

[14]  Breedveld Wentink,et al.  Eye-hand coordination in laparoscopy - an overview of experiments and supporting aids , 2001, Minimally invasive therapy & allied technologies : MITAT : official journal of the Society for Minimally Invasive Therapy.

[15]  Nobuhiko Hata,et al.  High-Resolution Stereoscopic Surgical Display Using Parallel Integral Videography and Multi-projector , 2002, MICCAI.

[16]  Ken Masamune,et al.  Three-Dimensional Slice Image Overlay System with Accurate Depth Perception for Surgery , 2000, MICCAI.

[17]  K. S. Arun,et al.  Least-Squares Fitting of Two 3-D Point Sets , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  J E Zöller,et al.  Image guided surgical navigation for removal of foreign bodies in the head and neck. , 2001, Journal of cranio-maxillo-facial surgery : official publication of the European Association for Cranio-Maxillo-Facial Surgery.

[19]  G. Stetten,et al.  Overlaying ultrasonographic images on direct vision. , 2001, Journal of ultrasound in medicine : official journal of the American Institute of Ultrasound in Medicine.

[20]  J.T. Lewis,et al.  An ultrasonic approach to localization of fiducial markers for interactive, image-guided neurosurgery. I. Principles , 1998, IEEE Transactions on Biomedical Engineering.

[21]  J. Berger,et al.  Computer-vision-enabled augmented reality fundus biomicroscopy. , 1999, Ophthalmology.

[22]  C Kremser,et al.  Image registration of MR and CT images using a frameless fiducial marker system. , 1997, Magnetic resonance imaging.

[23]  James A. Crowell,et al.  Horizontal and vertical disparity, eye position, and stereoscopic slant perception , 1999, Vision Research.

[24]  Robert J. Maciunas,et al.  Registration of head volume images using implantable fiducial markers , 1997, IEEE Transactions on Medical Imaging.

[25]  Henk G. Stassen,et al.  Theoretical background and conceptual solution for depth perception and eye-hand coordination problems in laparoscopic surgery , 1999 .

[26]  S Nakajima,et al.  Three-dimensional medical imaging display with computer-generated integral photography. , 2001, Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society.

[27]  H. Iseki,et al.  Endoscopic Augmented Reality Navigation System for Endonasal Transsphenoidal Surgery to Treat Pituitary Tumors: Technical Note , 2002, Neurosurgery.

[28]  Neil A. Dodgson,et al.  50-in. time-multiplexed autostereoscopic display , 2000, Electronic Imaging.