Volume Rendering of Multimodal Images for the Planning of Skull Base Surgery

In this paper we present an application in which multimodal images are used to generate pseudo-3D scenes to assist in the planning and execution of skull base surgery. Image data from CT, MRI and MR Angiography are registered such that the relationship between their image coordinates is known. The images are then transformed into occupancy maps for the structures of interest. Commercially available rendering algorithms then require that either the data from the multiple imaging modalities be combined into a single volumetric dataset prior to rendering, or that the data from each modality is rendered separately and subsequently overlaid. We present an algorithm that allows us to bypass this last stage and leave the objects in their original datasets. The 3D scenes are rendered by casting a ray simultaneously through the multi-dimensional space represented by the individual datasets.