PanoTrace: interactive 3D modeling of surround-view panoramic images in virtual reality

Full-surround panoramic imagery can provide a viewer with a high-resolution visual impression of a pictured real or realistically rendered environment, but it does not provide as high a level of immersion as modeled 3D geometry can, when viewed with virtual reality (VR) headsets or projection-based setups. In this paper, we demonstrate that augmenting panorama images with geometrical models can be done simply in VR itself and can significantly increase the feeling of immersion a viewer experiences. We propose a novel interactive modeling tool that allows users to model geometry depicted in a surround-panoramic scene directly in VR, utilizing projection mapping of the panorama on top of the evolving geometry. The user interface is intuitive and allows novice users to produce geometry that approximates ground truth models sufficiently to enhance a user's VR viewing experience. We designed a user study that compares users' self-reported levels of immersion, scene realism, and discomfort on a set of created models and comparison cases. Our results indicate that our modeled scenes produce a significantly higher sense of immersion than a basic dome geometry for the panorama when viewed in VR with head orientation and position tracking.