[POSTER] Holographic iRay: Exploring Augmentation for Medical Applications

A Holographic iRay prototype focusing on medical augmented reality is presented. The prototype is built using Microsoft HoloLens and Unity engine based on a previous iRay system built for iPad. A human subject is scanned using magnetic resonance imaging and the torso surface is pre-operatively segmented for 3D registration. The registration is performed using the iterative closest point algorithm between the pre-operative torso surface and the active torso surface mesh provided by the HoloLens spatial mapping and the gaze interaction. A scanning box is visualized at the gazing point to help the user select the targeting area. The pre-operative torso surface and the sampled active vertices within the scanning box are additionally overlaid in the box as the auxiliary information for guidance. Several simple interactions are designed to control the rendering of the inner organs after the registration. The experimental results demonstrate the potential of the Holographic iRay for medical applications.

[1]  Ioannis A. Kakadiaris,et al.  Semi-automatic Initial Registration for the iRay System: A User Study , 2017, AVR.

[2]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Ioannis A. Kakadiaris,et al.  iRay: Mobile AR Using Structure Sensor , 2016, 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct).