New Methods for Medical Augmented Reality

The increasing amount of imaging data in the operating room offers new possibilities for surgeons. Preoperative data like Computed Tomography and Magnetic Resonance Imaging allow for detailed anatomical information. Recently, intraoperative imaging like X-ray C-arms and ultrasound have become increasingly accessible in the OR and provide real-time anatomical images. Also, functional imaging such as PET and fMRI is accessible in a growing number of institutions. Corresponding intraoperative functional probes allow more efficient procedures to be introduced in the near future. The growing amount of imaging data increases the difficulty of retrieving the desired piece of information. Thus, efficient data representation becomes increasingly important for physicians. Visualization in computer assisted surgical solutions has not coped with the recent developments and does still not provide solutions which allow surgeons to take full advantage of the existing heterogeneous imaging data. Augmented Reality (AR) is a technology that has potential to improve physicians' performance by bringing the multitude of medical data, surgical actions, and patient into the same space. After more than a decade of research in medical Augmented Reality, the technology is sufficiently developed to create prototype systems for showrooms. However, it has still not found its way into operating rooms. This thesis addresses current issues of medical Augmented Reality in defining the right place, right time, and the right way of data representation. The first contribution is in providing an exhaustive state-of-the-art report on medical Augmented Reality. It then features a detailed overview of components that are necessary for a medical AR system and introduces a new method for temporal registration in AR. It provides details on the integration of these components and describes the necessary software framework that has been developed. The thesis presents new methods for assessment of a medical AR system addressing latency measurement and dynamic error prediction. The thesis concludes with validation which was done in close partnership with many surgeons.