Medical Image Volumetric Visualization: Algorithms, Pipelines, and Surgical Applications

With the increasing availability of high-resolution datasets of 3D medical images, the development of volumetric image rendering techniques have become an important complement to classical surface-based rendering. Since volumetric visualization does not require that surfaces be selected from within the 3D volumes, the full volume dataset is maintained during the rendering process. These methods are based on a foundation of projecting rays through volumes, which have a range of opacity attributes, onto a viewing window. Volume rendering is computationally demanding, and the ever increasing size of medical image datasets means that brute-force algorithms are not feasible for interactive use.

[1]  P. Hanrahan,et al.  Area and volume coherence for efficient visualization of 3D scalar functions , 1990, VVS.

[2]  Christof Rezk-Salama,et al.  High-Level User Interfaces for Transfer Function Design with Semantics , 2006, IEEE Transactions on Visualization and Computer Graphics.

[3]  A. James Stewart,et al.  Adaptive Slice Geometry for Hardware-Assisted Volume Rendering , 2005, J. Graph. Tools.

[4]  Lee Westover,et al.  Splatting: a parallel, feed-forward volume rendering algorithm , 1991 .

[5]  J. Lopera,et al.  Multidetector CT angiography of infrainguinal arterial bypass. , 2008, Radiographics : a review publication of the Radiological Society of North America, Inc.

[6]  Martin Kraus,et al.  Pre-Integrated Volume Rendering , 2005, The Visualization Handbook.

[7]  Min Chen,et al.  Feature Aligned Volume Manipulation for Illustration and Visualization , 2006, IEEE Transactions on Visualization and Computer Graphics.

[8]  R. Robb,et al.  3D visualization, analysis, and treatment of the prostate using trans-urethral ultrasound. , 2003, Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society.

[9]  Helko Lehmann,et al.  Fast maximum intensity projections of large medical data sets by exploiting hierarchical memory architectures , 2006, IEEE Transactions on Information Technology in Biomedicine.

[10]  Marc Levoy,et al.  Display of surfaces from volume data , 1988, IEEE Computer Graphics and Applications.

[11]  Georgios Sakas,et al.  Preprocessing and volume rendering of 3D ultrasonic data , 1995, IEEE Computer Graphics and Applications.

[12]  Andreas Pommert,et al.  3D-visualization of tomographic volume data using the generalized voxel-model , 1989, VVS.

[13]  Arie E. Kaufman,et al.  Texture Partitioning and Packing for Accelerating Texture-based Volume Rendering , 2003, Graphics Interface.

[14]  Kwan-Liu Ma,et al.  Hardware-accelerated parallel non-photorealistic volume rendering , 2002, NPAR '02.

[15]  David S. Ebert,et al.  Volume Illustration: Nonphotorealistic Rendering of Volume Models , 2001, IEEE Trans. Vis. Comput. Graph..

[16]  David L. Kao,et al.  An evaluation of using real-time volumetric display of 3D ultrasound data for intracardiac catheter manipulation tasks , 2005, Fourth International Workshop on Volume Graphics, 2005..

[17]  Klaus Mueller,et al.  A practical evaluation of popular volume rendering algorithms , 2000, VVS '00.

[18]  Ivan Viola,et al.  Illustrative visualization: new technology or useless tautology? , 2008, COMG.

[19]  M. Levoy,et al.  Fast volume rendering using a shear-warp factorization of the viewing transformation , 1994, SIGGRAPH.

[20]  Peter Hastreiter,et al.  High performance volume splatting for visualization of neurovascular data , 2005, VIS 05. IEEE Visualization, 2005..

[21]  Markus Hadwiger,et al.  High-quality two-level volume rendering of segmented data sets on consumer graphics hardware , 2003, IEEE Visualization, 2003. VIS 2003..

[22]  Jayaram K. Udupa,et al.  Shell rendering , 1993, IEEE Computer Graphics and Applications.

[23]  Bernd Hamann,et al.  Multiresolution techniques for interactive texture-based volume visualization , 1999, Proceedings Visualization '99 (Cat. No.99CB37067).

[24]  Stephen R. Aylward,et al.  Volume rendering of segmented image objects , 2002, IEEE Transactions on Medical Imaging.

[25]  Christof Rezk Salama.  Volumenvisualisierung auf handelsüblicher Grafik-Hardware (Volume Rendering Techniques for General Purpose Graphics Hardware) , 2005 .

[26]  Ricardo Marroquim,et al.  Volume and Isosurface Rendering with GPU‐Accelerated Cell Projection * , 2008, Comput. Graph. Forum.

[27]  Arie E. Kaufman,et al.  Accelerating volume rendering with texture hulls , 2002, Symposium on Volume Visualization and Graphics, 2002. Proceedings. IEEE / ACM SIGGRAPH.

[28]  D. DeLong,et al.  Two-dimensional multiplanar and three-dimensional volume-rendered vascular CT in pancreatic carcinoma: interobserver agreement and comparison with standard helical techniques. , 2001, AJR. American journal of roentgenology.

[29]  Daniel Cohen-Or,et al.  Volume graphics , 1993, Computer.

[30]  Eduard Gröller,et al.  Two-Level Volume Rendering , 2001, IEEE Trans. Vis. Comput. Graph..

[31]  Qi Zhang,et al.  Rapid Voxel Classification Methodology for Interactive 3D Medical Image Visualization , 2007, MICCAI.

[32]  Stefan Bruckner,et al.  Semantic Layers for Illustrative Volume Rendering , 2007, IEEE Transactions on Visualization and Computer Graphics.

[33]  S. Park,et al.  Linear polyp measurement at CT colonography: 3D endoluminal measurement with optimized surface-rendering threshold value and automated measurement. , 2008, Radiology.

[34]  Charl P. Botha,et al.  ShellSplatting: Interactive Rendering of Anisotropic Volumes , 2003, VisSym.

[35]  Nelson L. Max,et al.  Optical Models for Direct Volume Rendering , 1995, IEEE Trans. Vis. Comput. Graph..

[36]  Joe Michael Kniss,et al.  A Model for Volume Lighting and Modeling , 2003, IEEE Trans. Vis. Comput. Graph..

[37]  Ming-Yuen Chan,et al.  Relation-Aware Volume Exploration Pipeline , 2008, IEEE Transactions on Visualization and Computer Graphics.

[38]  Nipun Kwatra,et al.  Physics-Based Subsurface Visualization of Human Tissue , 2007 .

[39]  D. Sahani,et al.  Assessment of pancreatic tumor resectability with multidetector computed tomography: semiautomated console-generated images versus dedicated workstation-generated images. , 2008, Academic radiology.

[40]  Karl Heinz Höhne,et al.  Visualization blackboard: visualizing the visible human , 1996, IEEE Computer Graphics and Applications.

[41]  Allen Van Gelder,et al.  Direct volume rendering with shading via three-dimensional textures , 1996, Proceedings of 1996 Symposium on Volume Visualization.

[42]  Wolfgang Birkfellner,et al.  Fast DRR splat rendering using common consumer graphics hardware. , 2007, Medical physics.

[43]  Xiaoru Yuan,et al.  HDR VolVis: high dynamic range volume visualization , 2006, IEEE Transactions on Visualization and Computer Graphics.

[44]  Dani Tost,et al.  Multimodal volume rendering with 3D textures , 2008, Comput. Graph..

[45]  Kwan-Liu Ma,et al.  High-Quality Lighting for Pre-Integrated Volume Rendering , 2004, VisSym.

[46]  Susumu Shirayama,et al.  Numerical visualization by rapid isosurface extractions using 3D span spaces , 2008, J. Vis..

[47]  Martin Kraus,et al.  High-quality pre-integrated volume rendering using hardware-accelerated pixel shading , 2001, HWWS '01.

[48]  Jayaram K. Udupa,et al.  Artery-vein separation via MRA-An image processing approach , 2001, IEEE Transactions on Medical Imaging.

[49]  Klaus Mueller,et al.  GPU accelerated image aligned splatting , 2005, Fourth International Workshop on Volume Graphics, 2005..

[50]  Ö. Etlik,et al.  THREE-DIMENSIONAL VOLUME RENDERING IMAGING IN DETECTION OF BONE FRACTURES , 2004 .

[51]  Jian Huang,et al.  High-Quality Splatting on Rectilinear Grids with Efficient Culling of Occluded Voxels , 1999, IEEE Trans. Vis. Comput. Graph..

[52]  Markus Hadwiger,et al.  Real-time volume graphics , 2006, Eurographics.

[53]  Rüdiger Westermann,et al.  Acceleration techniques for GPU-based volume rendering , 2003, IEEE Visualization, 2003. VIS 2003..

[54]  Markus Hadwiger,et al.  Interactive Volume Exploration for Feature Detection and Quantification in Industrial CT Data , 2008, IEEE Transactions on Visualization and Computer Graphics.

[55]  Thomas Ertl,et al.  Interactive Clipping Techniques for Texture-Based Volume Visualization and Volume Shading , 2003, IEEE Trans. Vis. Comput. Graph..

[56]  P. Schellinger,et al.  Noninvasive Angiography (Magnetic Resonance and Computed Tomography) in the Diagnosis of Ischemic Cerebrovascular Disease , 2007, Cerebrovascular Diseases.

[57]  O. Ecabert,et al.  Visualizing the beating heart: interactive direct volume rendering of high-resolution CT time series using standard PC hardware , 2006, SPIE Medical Imaging.

[58]  A. Ahmetoğlu,et al.  MDCT cholangiography with volume rendering for the assessment of patients with biliary obstruction. , 2004, AJR. American journal of roentgenology.

[59]  Xiaoru Yuan,et al.  High dynamic range volume visualization , 2005, VIS 05. IEEE Visualization, 2005..

[60]  Martin Kraus,et al.  Hardware-accelerated volume and isosurface rendering based on cell-projection , 2000, Proceedings Visualization 2000. VIS 2000 (Cat. No.00CH37145).

[61]  Jayaram K. Udupa,et al.  Comparative analysis of shell rendering and shear-warp rendering , 2002, SPIE Medical Imaging.

[62]  Jae Young Lee,et al.  Three-dimensional imaging for hepatobiliary and pancreatic diseases: Emphasis on clinical utility , 2009, Indian Journal of Radiology and Imaging.

[63]  Markus Hadwiger,et al.  Real time computation and temporal coherence of opacity transfer functions for direct volume rendering of ultrasound data. , 2005, Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society.

[64]  F. Jacobson,et al.  MDCT evaluation of central airway and vascular complications of lung transplantation. , 2008, AJR. American journal of roentgenology.

[65]  Veysi Isler,et al.  Acceleration of direct volume rendering with programmable graphics hardware , 2006, The Visual Computer.

[66]  Eduard Gröller,et al.  Interactive High‐Quality Maximum Intensity Projection , 2000, Comput. Graph. Forum.

[67]  David Levin,et al.  Techniques for efficient, real-time, 3D visualization of multi-modality cardiac data using consumer graphics hardware. , 2005, Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society.

[68]  R. Shekhar,et al.  Cine MPR: interactive multiplanar reformatting of four-dimensional cardiac data using hardware-accelerated texture mapping , 2003, IEEE Transactions on Information Technology in Biomedicine.

[69]  Insung Ihm,et al.  On Enhancing the Speed of Splatting Using Both Object- and Image-Space Coherence , 2000, Graph. Model..

[70]  Brian Cabral,et al.  Accelerated volume rendering and tomographic reconstruction using texture mapping hardware , 1994, VVS '94.

[71]  Bernhard Preim,et al.  Visualization in Medicine: Theory, Algorithms, and Applications , 2007 .

[72]  Jan Kautz,et al.  Hardware Lighting and Shading: a Survey , 2004, Comput. Graph. Forum.

[73]  Santiago Martinez,et al.  MDCT angiography of thoracic aorta endovascular stent-grafts: pearls and pitfalls. , 2009, AJR. American journal of roentgenology.

[74]  Diego Borro,et al.  GPU Local Triangulation: an interpolating surface reconstruction algorithm , 2008, Comput. Graph. Forum.

[75]  Charles Hansen,et al.  The Visualization Handbook , 2011 .

[76]  Václav Skala,et al.  Space and time efficient isosurface extraction , 2008, Comput. Graph..

[77]  Thomas Ertl,et al.  Smart Hardware-Accelerated Volume Rendering , 2003, VisSym.

[78]  D. Ebert,et al.  Low-complexity maximum intensity projection , 2005, TOGS.

[79]  Carl-Fredrik Westin,et al.  Tissue Classification Based on 3D Local Intensity Structures for Volume Rendering , 2000, IEEE Trans. Vis. Comput. Graph..

[80]  Pat Hanrahan,et al.  Volume Rendering , 2020, Definitions.

[81]  R. Robb.  X-ray computed tomography: from basic principles to applications. , 1982, Annual review of biophysics and bioengineering.

[82]  Jayaram K. Udupa,et al.  T-shell rendering and manipulation , 2005, SPIE Medical Imaging.

[83]  James F. Blinn,et al.  Light reflection functions for simulation of clouds and dusty surfaces , 1982, SIGGRAPH.

[84]  Martin Kraus,et al.  Direct volume visualization of geometrically unpleasant meshes , 2003 .

[85]  E. Fishman,et al.  Volume rendering versus maximum intensity projection in CT angiography: what works best, when, and why. , 2006, Radiographics : a review publication of the Radiological Society of North America, Inc.

[86]  Lee Westover,et al.  Interactive volume rendering , 1989, VVS '89.

[87]  Markus Hadwiger,et al.  Real‐Time Ray‐Casting and Advanced Shading of Discrete Isosurfaces , 2005, Comput. Graph. Forum.

[88]  George Gentchos,et al.  CT and MR Angiography: Comprehensive Vascular Assessment , 2010 .

[89]  William E. Lorensen,et al.  The Transfer Function Bake-Off , 2001, IEEE Computer Graphics and Applications.

[90]  Wolfgang Birkfellner,et al.  Wobbled splatting—a fast perspective volume rendering method for simulation of x-ray images from CT , 2005, Physics in medicine and biology.

[91]  E. Fishman,et al.  Multidetector CT and three-dimensional CT angiography for suspected vascular trauma of the extremities. , 2008, Radiographics : a review publication of the Radiological Society of North America, Inc.

[92]  Joe Michael Kniss,et al.  Multidimensional Transfer Functions for Interactive Volume Rendering , 2002, IEEE Trans. Vis. Comput. Graph..

[93]  Christopher DeCoro,et al.  Advanced interactive medical visualization on the GPU , 2008, J. Parallel Distributed Comput..

[94]  Ulrich Neumann,et al.  Accelerating Volume Reconstruction With 3D Texture Hardware , 1994 .

[95]  David Salesin,et al.  Computer-generated pen-and-ink illustration , 1994, SIGGRAPH.

[96]  Klaus Mueller,et al.  Empty space skipping and occlusion clipping for texture-based volume rendering , 2003, IEEE Visualization, 2003. VIS 2003..

[97]  David H. Laidlaw,et al.  Interactive volume rendering of thin thread structures within multivalued scientific data sets , 2004, IEEE Transactions on Visualization and Computer Graphics.

[98]  William J. Schroeder,et al.  The Visualization Toolkit , 2005, The Visualization Handbook.

[99]  Heewon Kye,et al.  Accelerated MIP based on GPU using block clipping and occlusion query , 2008, Comput. Graph..

[100]  Alexandre X. Falcão,et al.  3D visualization to assist iterative object definition from medical images , 2006, Comput. Medical Imaging Graph..

[101]  Lee Westover,et al.  Footprint evaluation for volume rendering , 1990, SIGGRAPH.