Simultaneous graphics and multislice raster image display for interactive image-guided surgery

A surgeon typically uses information from a number of tomographic imaging methods (e.g., CT, MR, PET) during the course of a surgical procedure. These imaging techniques represent three-dimensional information as a set of two-dimensional images. To use this information, the surgeon is required to mentally construct a three-dimensional visualization from the set of two- dimensional images. The formation of the mental image becomes more complicated with the inclusion of multiple imaging modalities and multiple imaging planes. We have developed a technique to enhance the mental three-dimensional visualization process through simultaneous graphics and multislice raster image display. The composite display, capable of displaying up to three raster images along with a patient-specific graphics model, is viewed on a 1280 X 1024 monitor. The raster images, displayed in a 512 X 512 format, may be any combination of imaging methods and imaging planes. The graphics model, determined from the imaging data, may be freely rotated as a depth-cued wireframe or shaded-surface model. Regions-of-interest may be incorporated into the graphics model for additional visual cues. Trajectory information may be obtained by moving a three-dimensional cursor in any raster image space or in the graphics model with instantaneous update of the remaining display area. This design allows the surgeon to interactively obtain orientation and visualization information from the images in the operating room. Because the classic imaging planes are used, the surgeon is not required to deal with a new information format or a loss of resolution.