Evaluating surface visualization methods in semi-transparent volume rendering in virtual reality

ABSTRACT Perceptual visualisation of semi-transparent structures in volumetric datasets is challenging due to its inherent visual complexity. This is however of primary importance in medical visualisation where raymarching of volumetric data is common. While rendering volumetric data itself is a well-explored area, perception of volume-rendered images combined with semi-transparent mesh data remains an underexplored topic. As virtual reality (VR) is increasingly employed for medical data inspection, it becomes important to understand how different mesh visualisations affect performance in medical tasks in this context. When considering direct volume rendering in immersive VR, the effects of stereoscopic vision and motion parallax require additional consideration. We investigate how surface transparency modes affect task performance in VR. For two medical image analysis tasks, we conduct a user study (n = 23) to analyse the impact of mesh rendering methods on task outcome and subjective preference. Our evaluation indicates that user performance with wireframe rendering varies greatly between tasks while constant opacity and silhouette provide a stable benefit. Overall, transparent visualisations led to improved precision (lower error rate, lower inter-observer variability) and increased user confidence, while the efficiency of visualisation methods was task-dependent. This demonstrates how semi-transparent rendering will improve visual analysis in medical applications and other disciplines .

[1]  Timo Ropinski,et al.  A Survey of Volumetric Illumination Techniques for Interactive Volume Rendering , 2014, Comput. Graph. Forum.

[2]  Timo Ropinski,et al.  Historygrams: Enabling Interactive Global Illumination in Direct Volume Rendering using Photon Mapping , 2012, IEEE Transactions on Visualization and Computer Graphics.

[3]  Inna Tsirlin,et al.  Stereoscopic transparency: constraints on the perception of multiple surfaces. , 2008, Journal of vision.

[4]  S. Nagle,et al.  Inter- and intra-observer repeatability of aortic annulus measurements on screening CT for transcatheter aortic valve replacement (TAVR): Implications for appropriate device sizing. , 2018, European journal of radiology.

[5]  Markus Hadwiger,et al.  GPU-Based High-Quality Volume Rendering For Virtual Environments , 2006 .

[6]  Stefanie Speidel,et al.  IMHOTEP: virtual reality framework for surgical applications , 2018, International Journal of Computer Assisted Radiology and Surgery.

[7]  M. Gomez,et al.  INTEROBSERVER AND INTRAOBSERVER REPRODUCIBILITY OF AORTIC VALVAR COMPLEX MEASUREMENTS AND TRANSCATHETER AORTIC VALVE REPLACEMENT DEPLOYMENT ANGLES BY MULTIDETECTOR COMPUTED TOMOGRAPHY , 2014 .

[8]  Bart M. ter Haar Romeny,et al.  Flexible GPU-Based Multi-Volume Ray-Casting , 2008, VMV.

[9]  Timo Ropinski,et al.  Quantitative and Qualitative Analysis of the Perception of Semi‐Transparent Structures in Direct Volume Rendering , 2018, Comput. Graph. Forum.

[10]  Örjan Smedby,et al.  Stereoscopic static depth perception of enclosed 3D objects , 2013, SAP.

[11]  Rüdiger Westermann,et al.  Acceleration techniques for GPU-based volume rendering , 2003, IEEE Visualization, 2003. VIS 2003..

[12]  Timo Ropinski,et al.  Efficient volume illumination with multiple light sources through selective light updates , 2015, 2015 IEEE Pacific Visualization Symposium (PacificVis).

[13]  Philippe C. Cattin,et al.  Direct Visual and Haptic Volume Rendering of Medical Data Sets for an Immersive Exploration in Virtual Reality , 2019, MICCAI.

[14]  Dieter Schmalstieg,et al.  Liver Surgery Planning Using Virtual Reality , 2006, IEEE Computer Graphics and Applications.

[15]  Wa Wijnand IJsselsteijn,et al.  Effectiveness of stereoscopic displays in medicine: A review , 2012 .

[16]  Julie M. Harris,et al.  Volume perception: Disparity extraction and depth representation in complex three-dimensional environments. , 2014, Journal of vision.

[17]  Peter T. Tkacik,et al.  Color Schlieren imaging of high-pressure overexpanded planar nozzle flow using a simple, low-cost test apparatus , 2011, J. Vis..

[18]  Andrea Li,et al.  Perception of three-dimensional shape from texture is based on patterns of oriented energy , 2000, Vision Research.

[19]  Justin Hensley,et al.  Real‐Time Concurrent Linked List Construction on the GPU , 2010, Comput. Graph. Forum.

[20]  Brenton Keller,et al.  Real-time visualization and interaction with static and live optical coherence tomography volumes in immersive virtual reality. , 2018, Biomedical optics express.

[21]  Cass W. Everitt,et al.  Interactive Order-Independent Transparency , 2001 .

[22]  Marius Erdt,et al.  Single-Pass Stereoscopic GPU Ray Casting Using Re-Projection Layers , 2017, EGSR.

[23]  Danni Ai,et al.  Perception enhancement using importance-driven hybrid rendering for augmented reality based endoscopic surgical navigation. , 2018, Biomedical optics express.

[24]  Nital P. Appelbaum,et al.  The Application of Virtual Reality in Patient Education. , 2019, Annals of vascular surgery.

[25]  Stefan Bruckner,et al.  Interactive Dynamic Volume Illumination with Refraction and Caustics , 2018, IEEE Transactions on Visualization and Computer Graphics.

[26]  Louis Bavoil,et al.  Order Independent Transparency with Dual Depth Peeling , 2008 .

[27]  Mark Asselin,et al.  Interaction with Volume-Rendered Three-Dimensional Echocardiographic Images in Virtual Reality. , 2018, Journal of the American Society of Echocardiography : official publication of the American Society of Echocardiography.

[28]  Bernhard Preim,et al.  Perceptual Evaluation of Ghosted View Techniques for the Exploration of Vascular Structures and Embedded Flow , 2011, Comput. Graph. Forum.

[29]  Yun Jang,et al.  Smart Transparency for Illustrative Visualization of Complex Flow Surfaces , 2013, IEEE Transactions on Visualization and Computer Graphics.

[30]  G. Westheimer,et al.  Depth attraction and repulsion of disparate foveal stimuli , 1987, Vision Research.

[31]  Suzanne P. McKee,et al.  The spatial requirements for fine stereoacuity , 1983, Vision Research.

[32]  Boris M. Velichkovsky,et al.  The perception of egocentric distances in virtual environments - A review , 2013, ACM Comput. Surv..

[33]  Ulrich Eck,et al.  Visualization Techniques for Precise Alignment in VR: A Comparative Study , 2019, 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[34]  Timo Ropinski,et al.  Advanced illumination techniques for GPU-based volume raycasting , 2008, SIGGRAPH 2008.

[35]  Bernhard Preim,et al.  Combining Silhouettes, Surface, and Volume Rendering for Surgery Education and Planning , 2005, EuroVis.

[36]  Jonathan C. Silverstein,et al.  Precisely Exploring Medical Models and Volumes in Collaborative Virtual Reality , 2005, Presence: Teleoperators & Virtual Environments.