Feature of Interest‐Based Direct Volume Rendering Using Contextual Saliency‐Driven Ray Profile Analysis

Direct volume rendering (DVR) visualization helps interpretation because it allows users to focus attention on the subset of volumetric data that is of most interest to them. The ideal visualization of the features of interest (FOIs) in a volume, however, is still a major challenge. The clear depiction of FOIs depends on accurate identification of the FOIs and appropriate specification of the optical parameters via transfer function (TF) design and it is typically a repetitive trial‐and‐error process. We address this challenge by introducing a new method that uses contextual saliency information to group the voxels along a viewing ray into distinct FOIs where ‘contextual saliency’ is a biologically inspired attribute that aids the identification of features that the human visual system considers important. The saliency information is also used to automatically define the optical parameters that emphasize the visual depiction of the FOIs in DVR. We demonstrate the capabilities of our method by its application to a variety of volumetric data sets and highlight its advantages by comparison to current state‐of‐the‐art ray profile analysis methods.

[1]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Ali Borji,et al.  Quantitative Analysis of Human-Model Agreement in Visual Saliency Modeling: A Comparative Study , 2013, IEEE Transactions on Image Processing.

[3]  L. Griffeth Use of Pet/Ct Scanning in Cancer Patients: Technical and Practical Considerations , 2005, Proceedings.

[4]  Heinz Hügli,et al.  Assessing the contribution of color in visual attention , 2005, Comput. Vis. Image Underst..

[5]  Nathalie Guyader,et al.  Color information in a model of saliency , 2014, 2014 22nd European Signal Processing Conference (EUSIPCO).

[6]  Zhengqin Li,et al.  Superpixel segmentation using Linear Spectral Clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Sikun Li,et al.  Sketch-based interactive visualization: a survey , 2014, J. Vis..

[8]  David Dagan Feng,et al.  Efficient visibility-driven medical image visualisation via adaptive binned visibility histogram , 2016, Comput. Medical Imaging Graph..

[9]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[10]  Eduard Gröller,et al.  Feature peeling , 2007, GI '07.

[11]  Frans Vos,et al.  WYSIWYP: What You See Is What You Pick , 2012, IEEE Transactions on Visualization and Computer Graphics.

[12]  Ju Liu,et al.  Image Adaptation and Dynamic Browsing Based on Two-Layer Saliency Combination , 2013, IEEE Transactions on Broadcasting.

[13]  Joe Michael Kniss,et al.  Interactive volume rendering using multi-dimensional transfer functions and direct manipulation widgets , 2001, Proceedings Visualization, 2001. VIS '01..

[14]  Jørgen Arendt Jensen,et al.  Medical ultrasound imaging. , 2007, Progress in biophysics and molecular biology.

[15]  M. Sheelagh T. Carpendale,et al.  Color Tunneling: Interactive Exploration and Selection in Volumetric Datasets , 2014, 2014 IEEE Pacific Visualization Symposium.

[16]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[17]  Osman Ratib,et al.  OsiriX: An Open-Source Software for Navigating in Multidimensional DICOM Images , 2004, Journal of Digital Imaging.

[18]  Stefan Bruckner,et al.  Contextual picking of volumetric structures , 2009, 2009 IEEE Pacific Visualization Symposium.

[19]  Jean-Michel Dischler,et al.  Per-Pixel Opacity Modulation for Feature Enhancement in Volume Rendering , 2010, IEEE Transactions on Visualization and Computer Graphics.

[20]  Samuel S. Silva,et al.  Using color in visualization: A survey , 2011, Comput. Graph..

[21]  Anders Ynnerman,et al.  Local Histograms for Design of Transfer Functions in Direct Volume Rendering , 2006, IEEE Transactions on Visualization and Computer Graphics.

[22]  Zaid J. Towfic,et al.  The Lung Image Database Consortium (LIDC) data collection process for nodule detection and annotation , 2007, SPIE Medical Imaging.

[23]  D. Ballard,et al.  Eye movements in natural behavior , 2005, Trends in Cognitive Sciences.

[24]  Yingcai Wu,et al.  Interactive Transfer Function Design Based on Editing Direct Volume Rendered Images , 2007, IEEE Transactions on Visualization and Computer Graphics.

[25]  Timo Ropinski,et al.  Voreen: A Rapid-Prototyping Environment for Ray-Casting-Based Volume Visualizations , 2009, IEEE Computer Graphics and Applications.

[26]  John Dingliana,et al.  Visibility-Weighted Saliency for Volume Visualization , 2015, CGVC.

[27]  Susan L. Franzel,et al.  Guided search: an alternative to the feature integration model for visual search. , 1989, Journal of experimental psychology. Human perception and performance.

[28]  A. G. McKendrick,et al.  XLV.—The Rate of Multiplication of Micro-organisms: A Mathematical Study , 1912 .

[29]  R. Hanajima,et al.  Where Do Neurologists Look When Viewing Brain CT Images? An Eye-Tracking Study Involving Stroke Cases , 2011, PloS one.

[30]  L. Itti,et al.  Modeling the influence of task on attention , 2005, Vision Research.

[31]  David W. Jacobs,et al.  Mesh saliency , 2005, SIGGRAPH 2005.

[32]  Silvia Castroz,et al.  Transfer Function Specification for the Visualization of Medical Data , 2006 .

[33]  Gordon Clapworthy,et al.  Accumulation of local maximum intensity for feature enhanced volume rendering , 2012, The Visual Computer.

[34]  Ivan Viola,et al.  Importance-driven feature enhancement in volume visualization , 2005, IEEE Transactions on Visualization and Computer Graphics.

[35]  Kwan-Liu Ma,et al.  RGVis: region growing based techniques for volume visualization , 2003, 11th Pacific Conference onComputer Graphics and Applications, 2003. Proceedings..

[36]  P Reinagel,et al.  Natural scene statistics at the centre of gaze. , 1999, Network.

[37]  David S. Ebert,et al.  Feature-Driven Data Exploration for Volumetric Rendering , 2012, IEEE Transactions on Visualization and Computer Graphics.

[38]  Laurent Itti,et al.  Automatic foveation for video compression using a neurobiological model of visual attention , 2004, IEEE Transactions on Image Processing.

[39]  Brendan J. Frey,et al.  Factor graphs and the sum-product algorithm , 2001, IEEE Trans. Inf. Theory.

[40]  Cynthia A. Brewer,et al.  ColorBrewer.org: An Online Tool for Selecting Colour Schemes for Maps , 2003 .

[41]  Herbert Freeman,et al.  Machine Vision for Three-Dimensional Scenes , 1990 .

[42]  Tai Sing Lee,et al.  Hierarchical Bayesian inference in the visual cortex. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[43]  Stefan Bruckner,et al.  Instant Volume Visualization using Maximum Intensity Difference Accumulation , 2009, Comput. Graph. Forum.

[44]  J. van Leeuwen,et al.  Neural Networks: Tricks of the Trade , 2002, Lecture Notes in Computer Science.

[45]  Laird Ak Dynamics of Tumour Growth , 1964 .

[46]  Michael F. McNitt-Gray,et al.  The Lung Image Database Consortium (LIDC) data collection process for nodule detection and annotation , 2007, SPIE Medical Imaging.

[47]  Klaus-Robert Müller,et al.  Efficient BackProp , 2012, Neural Networks: Tricks of the Trade.

[48]  Kwan-Liu Ma,et al.  A novel interface for higher-dimensional classification of volume data , 2003, IEEE Visualization, 2003. VIS 2003..

[49]  Kwan-Liu Ma,et al.  Size-based Transfer Functions: A New Volume Exploration Technique , 2008, IEEE Transactions on Visualization and Computer Graphics.

[50]  William E. Lorensen,et al.  The Transfer Function Bake-Off , 2001, IEEE Computer Graphics and Applications.

[51]  C. Frith,et al.  Directing attention to locations and to sensory modalities: multiple levels of selective processing revealed with PET. , 2002, Cerebral cortex.

[52]  Nelson L. Max,et al.  Progress in scientific visualization , 2005, The Visual Computer.

[53]  BorjiAli,et al.  State-of-the-Art in Visual Attention Modeling , 2013 .

[54]  Hanspeter Pfister,et al.  Generation of transfer functions with stochastic search techniques , 1996, Proceedings of Seventh Annual IEEE Visualization '96.

[55]  A. K. Laird Dynamics of Tumour Growth , 1964, British Journal of Cancer.

[56]  X. Jin Factor graphs and the Sum-Product Algorithm , 2002 .

[57]  Pietro Perona,et al.  Optimal reward harvesting in complex perceptual environments , 2010, Proceedings of the National Academy of Sciences.

[58]  Li Xu,et al.  Hierarchical Saliency Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[59]  Penny Rheingans,et al.  Texture-based Transfer Functions for Direct Volume Rendering , 2008, IEEE Transactions on Visualization and Computer Graphics.

[60]  Harry Shum,et al.  Picture Collage , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[61]  Liang-Tien Chia,et al.  Region-Based Saliency Detection and Its Application in Object Recognition , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[62]  David Dagan Feng,et al.  Visibility-driven PET-CT visualisation with region of interest (ROI) segmentation , 2013, The Visual Computer.

[63]  Jian Zhang,et al.  Automating Transfer Function Design with Valley Cell‐Based Clustering of 2D Density Plots , 2012, Comput. Graph. Forum.

[64]  David W. Jacobs,et al.  Mesh saliency , 2005, ACM Trans. Graph..

[65]  Jim R. Parker,et al.  Algorithms for image processing and computer vision , 1996 .

[66]  Sinisa Pajevic,et al.  Color schemes to represent the orientation of anisotropic tissues from diffusion tensor data: Application to white matter fiber tract mapping in the human brain , 1999, Magnetic resonance in medicine.

[67]  Jian Zhang,et al.  Efficient Volume Exploration Using the Gaussian Mixture Model , 2011, IEEE Transactions on Visualization and Computer Graphics.

[68]  Xiaoru Yuan,et al.  WYSIWYG (What You See is What You Get) Volume Visualization , 2011, IEEE Transactions on Visualization and Computer Graphics.

[69]  David Dagan Feng,et al.  Saliency-Based Lesion Segmentation Via Background Detection in Dermoscopic Images , 2017, IEEE Journal of Biomedical and Health Informatics.

[70]  Han-Wei Shen,et al.  View selection for volume rendering , 2005, VIS 05. IEEE Visualization, 2005..

[71]  Sukhyun Lim,et al.  GPU‐based interactive visualization framework for ultrasound datasets , 2009, Comput. Animat. Virtual Worlds.

[72]  Claudio M. Privitera,et al.  Algorithms for Defining Visual Regions-of-Interest: Comparison with Eye Fixations , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[73]  Adam Finkelstein,et al.  WYSIWYG NPR: drawing strokes directly on 3D models , 2002, SIGGRAPH.

[74]  Helwig Hauser,et al.  Visualization of Multi‐Variate Scientific Data , 2009, Comput. Graph. Forum.

[75]  Ali Borji,et al.  Salient Object Detection: A Benchmark , 2015, IEEE Transactions on Image Processing.

[76]  Pietro Perona,et al.  Is bottom-up attention useful for object recognition? , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[77]  Subhashis Banerjee,et al.  A Novel GBM Saliency Detection Model Using Multi-Channel MRI , 2016, PloS one.

[78]  Sikun Li,et al.  Spatiotemporal volume saliency , 2016, J. Vis..

[79]  Eduard Gröller,et al.  Two-Level Volume Rendering , 2001, IEEE Trans. Vis. Comput. Graph..

[80]  Jörg-Stefan Praßni,et al.  Stroke-Based Transfer Function Design , 2008, VG/PBG@SIGGRAPH.

[81]  Kwan-Liu Ma,et al.  Visibility Histograms and Visibility-Driven Transfer Functions , 2011, IEEE Transactions on Visualization and Computer Graphics.

[82]  Amitabh Varshney,et al.  Saliency-guided Enhancement for Volume Visualization , 2006, IEEE Transactions on Visualization and Computer Graphics.

[83]  David Dagan Feng,et al.  Occlusion and Slice-Based Volume Rendering Augmentation for PET-CT , 2017, IEEE Journal of Biomedical and Health Informatics.

[84]  Laird Ak DYNAMICS OF TUMOR GROWTH. , 1964 .