Proposal and Evaluation of AR-Based Microscopic Brain Surgery Support System

In this study, we describe an augmented reality (AR) support system that superimposes a three-dimensional computer graphics (3DCG) brain model image on an intraoperative microscopic image using simultaneous localization and mapping (SLAM). First, we extract regions such as tumor and blood vessel from computed tomography (CT) images or magnetic resonance images (MRI) of real patients, where data is stored in the digital imaging and communications in medicine (DICOM) format. Then, the entire brain and identified regions are converted into three-dimensional computer graphics (3DCG) model with a standard triangulated language (STL) format. The motion of the camera is estimated from the surgical video using SLAM. Based on the movement, the 3DCG brain model can be moved according to the movement of the real brain. Finally, we verified the estimation accuracy of the camera movement using SLAM.