Stereo Augmented Reality in the Surgical Microscope

This paper describes the MAGI (microscope-assisted guided interventions) augmented-reality system, which allows surgeons to view virtual features segmented from preoperative radiological images accurately overlaid in stereo in the optical path of a surgical microscope. The aim of the system is to enable the surgeon to see in the correct 3-D position the structures that are beneath the physical surface. The technical challenges involved are calibration, segmentation, registration, tracking, and visualization. This paper details our solutions to these problems. As it is difficult to make reliable quantitative assessments of the accuracy of augmented-reality systems, results are presented from a numerical simulation, and these show that the system has a theoretical overlay accuracy of better than 1 mm at the focal plane of the microscope. Implementations of the system have been tested on volunteers, phantoms, and seven patients in the operating room. Observations are consistent with this accuracy prediction.

[1]  William Schroeder,et al.  The Visualization Toolkit: An Object-Oriented Approach to 3-D Graphics , 1997 .

[2]  LensesReg G. Willson.  Modeling and Calibration of Automated Zoom , 1994 .

[3]  Derek L. G. Hill,et al.  Estimation of intraoperative brain surface movement , 1997, CVRMed.

[4]  Richard L. Holloway,et al.  Registration Error Analysis for Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[5]  Jay B. West,et al.  Predicting error in rigid-body point-based registration , 1998, IEEE Transactions on Medical Imaging.

[6]  J. Strohbehn,et al.  A frameless stereotaxic integration of computerized tomographic imaging and the operating microscope. , 1986, Journal of neurosurgery.

[7]  R Probst,et al.  Noninvasive Tracking of Patient's Head Movements During Computer‐Assisted Intranasal Microscopic Surgery , 1997, The Laryngoscope.

[8]  Colin Studholme,et al.  An overlap invariant entropy measure of 3D medical image alignment , 1999, Pattern Recognit..

[9]  P. Kelly,et al.  Computer-assisted stereotactic microsurgery for the treatment of intracranial neoplasms. , 1982, Neurosurgery.

[10]  Robert J. Maciunas,et al.  Registration of head volume images using implantable fiducial markers , 1997, IEEE Transactions on Medical Imaging.

[11]  D. Hill,et al.  Augmentation of reality using an operating microscope for otolaryngology and neurosurgical guidance. , 1995, Journal of image guided surgery.

[12]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..