GPU-accelerated Rendering for Medical Augmented Reality in Minimally-invasive Procedures

Recent advances in GPU programmability and performance have enabled development of real-time high quality volume visualization algorithms. Medical augmented reality systems can benefit from these developments. Task-specific visualization aids physicians in better understanding the patient’s anatomy and supports navigation of medical instruments in absence of a direct line of sight in minimally-invasive procedures. In this paper we present our results of integration of a hardware accelerated volume renderer into a medical augmented reality framework using a video see-through head mounted display (HMD). The performance of the system is evaluated in an experiment for two human CT datasets. Compared to the literature, our approach allows direct realtime stereo visualization of volumetric medical data on a HMD without prior time consuming pre-processing or segmentation. To further improve the visual perception and interaction of real and virtual objects, the renderer implements a virtual mirror and occlusion handling with the physicians hands and tracked medical instruments.