Dynamic real-time 4D cardiac MDCT image display using GPU-accelerated volume rendering

Intraoperative cardiac monitoring, accurate preoperative diagnosis, and surgical planning are important components of minimally-invasive cardiac therapy. Retrospective, electrocardiographically (ECG) gated, multidetector computed tomographical (MDCT), four-dimensional (3D + time), real-time, cardiac image visualization is an important tool for the surgeon in such procedure, particularly if the dynamic volumetric image can be registered to, and fused with the actual patient anatomy. The addition of stereoscopic imaging provides a more intuitive environment by adding binocular vision and depth cues to structures within the beating heart. In this paper, we describe the design and implementation of a comprehensive stereoscopic 4D cardiac image visualization and manipulation platform, based on the opacity density radiation model, which exploits the power of modern graphics processing units (GPUs) in the rendering pipeline. In addition, we present a new algorithm to synchronize the phases of the dynamic heart to clinical ECG signals, and to calculate and compensate for latencies in the visualization pipeline. A dynamic multiresolution display is implemented to enable the interactive selection and emphasis of volume of interest (VOI) within the entire contextual cardiac volume and to enhance performance, and a novel color and opacity adjustment algorithm is designed to increase the uniformity of the rendered multiresolution image of heart. Our system provides a visualization environment superior to noninteractive software-based implementations, but with a rendering speed that is comparable to traditional, but inferior quality, volume rendering approaches based on texture mapping. This retrospective ECG-gated dynamic cardiac display system can provide real-time feedback regarding the suspected pathology, function, and structural defects, as well as anatomical information such as chamber volume and morphology.

[1]  O. Ecabert,et al.  Visualizing the beating heart: interactive direct volume rendering of high-resolution CT time series using standard PC hardware , 2006, SPIE Medical Imaging.

[2]  Terry Peters,et al.  Dose reduction for cardiac CT using a registration-based approach. , 2007, Medical physics.

[3]  David L. Kao,et al.  An evaluation of using real-time volumetric display of 3D ultrasound data for intracardiac catheter manipulation tasks , 2005, Fourth International Workshop on Volume Graphics, 2005..

[4]  Matt Pharr,et al.  Gpu gems 2: programming techniques for high-performance graphics and general-purpose computation , 2005 .

[5]  Douglas L. Jones,et al.  En bloc exclusion of the pulmonary vein region in the pig using off pump, beating, intra-cardiac surgery: a pilot study of minimally invasive surgery for atrial fibrillation. , 2005, The Annals of thoracic surgery.

[6]  Qi Zhang,et al.  Graphics hardware based volumetric medical dataset visualization and classification , 2006, SPIE Medical Imaging.

[7]  Douglas L. Jones,et al.  Mitral valve implantation using off-pump closed beating intracardiac surgery: a feasibility study. , 2007, Interactive cardiovascular and thoracic surgery.

[8]  Nassir Navab,et al.  Depth Perception - A Major Issue in Medical AR: Evaluation Study by Twenty Surgeons , 2006, MICCAI.

[9]  Jayaram K. Udupa,et al.  Shell rendering , 1993, IEEE Computer Graphics and Applications.

[10]  Eduard Gröller,et al.  Maximum intensity projection at warp speed , 2000, Comput. Graph..

[11]  Thomas Flohr,et al.  Multislice helical CT of the heart with retrospective ECG gating: reduction of radiation exposure by ECG-controlled tube current modulation , 2002, European Radiology.

[12]  Nassir Navab,et al.  Action- and Workflow-Driven Augmented Reality for Computer-Aided Medical Procedures , 2007, IEEE Computer Graphics and Applications.

[13]  Kuni Ohtomo,et al.  Real-time four-dimensional imaging of the heart with multi-detector row CT. , 2003, Radiographics : a review publication of the Radiological Society of North America, Inc.

[14]  Ulrich Neumann,et al.  Accelerating Volume Reconstruction With 3D Texture Hardware , 1994 .

[15]  Mark Hastenteufel,et al.  Image‐based guidance for minimally invasive surgical atrial fibrillation ablation , 2006, The international journal of medical robotics + computer assisted surgery : MRCAS.

[16]  David Levin,et al.  Techniques for efficient, real-time, 3D visualization of multi-modality cardiac data using consumer graphics hardware. , 2005, Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society.

[17]  R. Shekhar,et al.  Cine MPR: interactive multiplanar reformatting of four-dimensional cardiac data using hardware-accelerated texture mapping , 2003, IEEE Transactions on Information Technology in Biomedicine.

[18]  James F. Blinn,et al.  Light reflection functions for simulation of clouds and dusty surfaces , 1982, SIGGRAPH.

[19]  Philippe Lacroute,et al.  Analysis of a Parallel Volume Rendering System Based on the Shear-Warp Factorization , 1996, IEEE Trans. Vis. Comput. Graph..

[20]  Naga K. Govindaraju,et al.  A Survey of General‐Purpose Computation on Graphics Hardware , 2007 .

[21]  R. Robb,et al.  Biomedical image visualization research using the Visible Human Datasets , 2006, Clinical anatomy.

[22]  Terry M. Peters,et al.  Simulation of Intra-operative 3D Coronary Angiography for Enhanced Minimally Invasive Robotic Cardiac Intervention , 2002, MICCAI.

[23]  Douglas L. Jones,et al.  An augmented reality environment for image-guidance of off-pump mitral valve implantation , 2007, SPIE Medical Imaging.

[24]  Terry M. Peters,et al.  Validation of dynamic heart models obtained using non-linear registration for virtual reality training, planning, and guidance of minimally invasive cardiac surgeries , 2004, Medical Image Anal..

[25]  Terry M. Peters,et al.  Towards Subject-Specific Models of the Dynamic Heart for Image-Guided Mitral Valve Surgery , 2007, MICCAI.

[26]  Joe Michael Kniss,et al.  Interactive Texture-Based Volume Rendering for Large Data Sets , 2001, IEEE Computer Graphics and Applications.

[27]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[28]  Roberto Scopigno,et al.  Multiresolution volume visualization with a texture-based octree , 2001, The Visual Computer.

[29]  Damien Maupu,et al.  3D stereo interactive medical visualization , 2005, IEEE Computer Graphics and Applications.

[30]  Alexandre X. Falcão,et al.  3D visualization to assist iterative object definition from medical images , 2006, Comput. Medical Imaging Graph..

[31]  Matthias Zwicker,et al.  3 Ideal Resampling 3 . 1 Sampling and Aliasing , 2022 .

[32]  Jayaram K. Udupa,et al.  Artery-vein separation via MRA-An image processing approach , 2001, IEEE Transactions on Medical Imaging.

[33]  Qi Zhang,et al.  Real-time visualization of 4D cardiac MR images using graphics processing units , 2006, 3rd IEEE International Symposium on Biomedical Imaging: Nano to Macro, 2006..

[34]  Marc Levoy,et al.  Display of surfaces from volume data , 1988, IEEE Computer Graphics and Applications.

[35]  B. Ohnesorge,et al.  Developments in CT , 2006 .

[36]  Heinrich Niemann,et al.  Local 3D Reconstruction and Augmented Reality Visualization of Free-Hand Ultrasound for Needle Biopsy Procedures , 2003, MICCAI.

[37]  Terry M. Peters,et al.  Optimal display conditions for quantitative analysis of stereoscopic cerebral angiograms , 1996, IEEE Trans. Medical Imaging.

[38]  Terry M. Peters,et al.  Automatic fusion of freehand endoscopic brain images to three-dimensional surfaces: creating stereoscopic panoramas , 2002, IEEE Transactions on Medical Imaging.

[39]  Pat Hanrahan,et al.  Hierarchical splatting: a progressive refinement algorithm for volume rendering , 1991, SIGGRAPH.

[40]  Suresh Senan,et al.  Use of maximum intensity projections (MIP) for target volume generation in 4DCT scans for lung cancer. , 2005, International journal of radiation oncology, biology, physics.

[41]  Rüdiger Westermann,et al.  Acceleration techniques for GPU-based volume rendering , 2003, IEEE Visualization, 2003. VIS 2003..

[42]  T M Peters,et al.  Integration of stereoscopic DSA and 3D MRI for image-guided neurosurgery. , 1994, Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society.

[43]  Xiao Hui Wang,et al.  Real-time stereographic rendering and display of medical images with programmable GPUs , 2008, Comput. Medical Imaging Graph..

[44]  Leo P Lawler,et al.  Four-dimensional imaging of the heart based on near-isotropic MDCT data sets. , 2005, AJR. American journal of roentgenology.

[45]  Herbert Bruder,et al.  Advances in cardiac imaging with 16-section CT systems. , 2003, Academic radiology.

[46]  Mahadevappa Mahesh,et al.  Physics of cardiac imaging with multiple-row detector CT. , 2007, Radiographics : a review publication of the Radiological Society of North America, Inc.

[47]  Andreas Pommert,et al.  3D visualization of tomographic volume data using the generalized voxel model , 1990, VVS '89.

[48]  D. DeLong,et al.  Two-dimensional multiplanar and three-dimensional volume-rendered vascular CT in pancreatic carcinoma: interobserver agreement and comparison with standard helical techniques. , 2001, AJR. American journal of roentgenology.

[49]  Chaoli Wang,et al.  Interactive Level-of-Detail Selection Using Image-Based Quality Metric for Large Volume Visualization , 2007, IEEE Transactions on Visualization and Computer Graphics.

[50]  R. Robb,et al.  3D visualization, analysis, and treatment of the prostate using trans-urethral ultrasound. , 2003, Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society.