Interaction Driven Enhancement of Depth Perception in Angiographic Volumes

User interaction has the potential to greatly facilitate the exploration and understanding of 3D medical images for diagnosis and treatment. However, in certain specialized environments such as in an operating room (OR), technical and physical constraints such as the need to enforce strict sterility rules, make interaction challenging. In this paper, we propose to facilitate the intraoperative exploration of angiographic volumes by leveraging the motion of a tracked surgical pointer, a tool that is already manipulated by the surgeon when using a navigation system in the OR. We designed and implemented three interactive rendering techniques based on this principle. The benefit of each of these techniques is compared to its non-interactive counterpart in a psychophysics experiment where 20 medical imaging experts were asked to perform a reaching/targeting task while visualizing a 3D volume of angiographic data. The study showed a significant improvement of the appreciation of local vascular structure when using dynamic techniques, while not having a negative impact on the appreciation of the global structure and only a marginal impact on the execution speed. A qualitative evaluation of the different techniques showed a preference for dynamic chroma-depth in accordance with the objective metrics but a discrepancy between objective and subjective measures for dynamic aerial perspective and shading.

[1]  Simon Drouin,et al.  PRISM: An open source framework for the interactive design of GPU volume rendering shaders , 2018, PloS one.

[2]  David J. Hawkes,et al.  Design and evaluation of a system for microscope-assisted guided interventions (MAGI) , 1999, IEEE Transactions on Medical Imaging.

[3]  K. May,et al.  Chromostereopsis: a multicomponent depth effect? , 1993 .

[4]  Dieter Schmalstieg,et al.  Interactive context-driven visualization tools for augmented reality , 2006, 2006 IEEE/ACM International Symposium on Mixed and Augmented Reality.

[5]  O. Bock,et al.  Mechanisms for sensorimotor adaptation to rotated visual input , 2001, Experimental Brain Research.

[6]  Bernhard Preim,et al.  Real-Time Illustration of Vascular Structures , 2006, IEEE Transactions on Visualization and Computer Graphics.

[7]  Dustin Scheinost,et al.  Novel interaction techniques for neurosurgical planning and stereotactic navigation , 2008, IEEE Transactions on Visualization and Computer Graphics.

[8]  Nassir Navab,et al.  Contextual Anatomic Mimesis Hybrid In-Situ Visualization Method for Improving Multi-Sensory Depth Perception in Medical Augmented Reality , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[9]  Terry Peters,et al.  Perceptual enhancement of arteriovenous malformation in MRI angiography displays , 2012, Medical Imaging.

[10]  Michael J. Bailey,et al.  Using ChromaDepth to Obtain Inexpensive Single-image Stereovision for Scientific Visualization , 1998, J. Graphics, GPU, & Game Tools.

[11]  Nassir Navab,et al.  Improved interventional X-ray appearance , 2014, 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[12]  G. Sandini,et al.  Eye-Hand Coordination during Dynamic Visuomotor Rotations , 2009, PloS one.

[13]  Timo Ropinski,et al.  A Survey of Perceptually Motivated 3D Visualization of Medical Image Data , 2016, Comput. Graph. Forum.

[14]  Kai Lawonn,et al.  Improving spatial perception of vascular models using supporting anchors and illustrative visualization , 2017, Comput. Graph..

[15]  J. Koenderink,et al.  Surface perception in pictures , 1992, Perception & psychophysics.

[16]  J. Edward Swan,et al.  Depth judgment measures and occluding surfaces in near-field augmented reality , 2010, APGV '10.

[17]  Timo Ropinski,et al.  Visually Supporting Depth Perception in Angiography Imaging , 2006, Smart Graphics.

[18]  Kai Lawonn,et al.  Streamlines for Illustrative Real‐Time Rendering , 2013, Comput. Graph. Forum.

[19]  Abbas F. Sadikot,et al.  IBIS: an OR ready open-source platform for image-guided neurosurgery , 2017, International Journal of Computer Assisted Radiology and Surgery.

[20]  Xiaoning Qian,et al.  Effective visualization of complex vascular structures using a non-parametric vessel detection method , 2008, IEEE Transactions on Visualization and Computer Graphics.

[21]  Bernard D. Adelstein,et al.  Localization of a Time-Delayed, Monocular Virtual Object Superimposed on a Real Environment , 2000, Presence: Teleoperators & Virtual Environments.

[22]  Mark Wexler,et al.  Depth perception by the active observer , 2005, Trends in Cognitive Sciences.

[23]  C. S. Harris Perceptual adaptation to inverted, reversed, and displaced vision. , 1965, Psychological review.

[24]  Lejing Wang,et al.  Augmented depth perception visualization in 2D/3D image fusion , 2014, Comput. Medical Imaging Graph..

[25]  Stephen R. Ellis,et al.  Localization of Virtual Objects in the Near Visual Field , 1998, Hum. Factors.

[26]  Dieter Schmalstieg,et al.  Comprehensible Visualization for Augmented Reality , 2009, IEEE Transactions on Visualization and Computer Graphics.

[27]  Kai Lawonn,et al.  Illustrative Visualization of Vascular Models for Static 2D Representations , 2015, MICCAI.

[28]  Nassir Navab,et al.  The Virtual Mirror: A New Interaction Paradigm for Augmented Reality Environments , 2009, IEEE Transactions on Medical Imaging.

[29]  James F. Blinn,et al.  Models of light reflection for computer synthesized pictures , 1977, SIGGRAPH.

[30]  Jannick P. Rolland,et al.  Towards Quantifying Depth and Size Perception in Virtual Environments , 1993, Presence: Teleoperators & Virtual Environments.

[31]  Guang-Zhong Yang,et al.  Visual Force Feedback for Hand-Held Microsurgical Instruments , 2015, MICCAI.

[32]  D. Louis Collins,et al.  An Evaluation of Depth Enhancing Perceptual Cues for Vascular Volume Visualization in Neurosurgery , 2014, IEEE Transactions on Visualization and Computer Graphics.

[33]  Timo Ropinski,et al.  Void Space Surfaces to Convey Depth in Vessel Visualizations , 2020, IEEE transactions on visualization and computer graphics.

[34]  R. Gregory Eye and Brain: The Psychology of Seeing , 1966 .

[35]  A. James Stewart,et al.  Direct Surgeon Control of the Computer in the Operating Room , 2011, MICCAI.

[36]  G P Bingham,et al.  Accommodation, occlusion, and disparity matching are used to guide reaching: a comparison of actual versus virtual environments. , 2001, Journal of experimental psychology. Human perception and performance.

[37]  Jannick P. Rolland,et al.  Method of Adjustments versus Method of Constant Stimuli in the Quantification of Accuracy and Precision of Rendered Depth in Head-Mounted Displays , 2002, Presence: Teleoperators & Virtual Environments.