Medical image segmentation on GPUs - A comprehensive review

Segmentation of anatomical structures, from modalities like computed tomography (CT), magnetic resonance imaging (MRI) and ultrasound, is a key enabling technology for medical applications such as diagnostics, planning and guidance. More efficient implementations are necessary, as most segmentation methods are computationally expensive, and the amount of medical imaging data is growing. The increased programmability of graphic processing units (GPUs) in recent years have enabled their use in several areas. GPUs can solve large data parallel problems at a higher speed than the traditional CPU, while being more affordable and energy efficient than distributed systems. Furthermore, using a GPU enables concurrent visualization and interactive segmentation, where the user can help the algorithm to achieve a satisfactory result. This review investigates the use of GPUs to accelerate medical image segmentation methods. A set of criteria for efficient use of GPUs are defined and each segmentation method is rated accordingly. In addition, references to relevant GPU implementations and insight into GPU optimization are provided and discussed. The review concludes that most segmentation methods may benefit from GPU processing due to the methods' data parallel structure and high thread count. However, factors such as synchronization, branch divergence and memory usage can limit the speedup.

[1]  MengChu Zhou,et al.  Image Ratio Features for Facial Expression Recognition Application , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[2]  Lei Pan,et al.  Implementation of medical image segmentation in CUDA , 2008, 2008 International Conference on Information Technology and Applications in Biomedicine.

[3]  Francis K. H. Quek,et al.  A review of vessel extraction techniques and algorithms , 2004, CSUR.

[4]  Juan José Pantrigo,et al.  Bandwidth-Improved GPU Particle Filter for Visual Tracking , 2006 .

[5]  Nick Barnes,et al.  Speeding up Mutual Information Computation Using NVIDIA CUDA Hardware , 2007, 9th Biennial Conference of the Australian Pattern Recognition Society on Digital Image Computing Techniques and Applications (DICTA 2007).

[6]  Giorgio Panin,et al.  Model-based Visual Tracking: The OpenTL Framework , 2011 .

[7]  Pradeep Dubey,et al.  Debunking the 100X GPU vs. CPU myth: an evaluation of throughput computing on CPU and GPU , 2010, ISCA.

[8]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[9]  Rodney A. Kennedy,et al.  Parallel computation of mutual information on the GPU with application to real-time registration of 3D medical images , 2010, Comput. Methods Programs Biomed..

[10]  Hideo Saito,et al.  Real-Time Online Video Object Silhouette Extraction Using Graph Cuts on the GPU , 2009, ICIAP.

[11]  Ross T. Whitaker,et al.  GIST: an interactive, GPU-based level set segmentation tool for 3D medical images , 2004, Medical Image Anal..

[12]  Rüdiger Westermann,et al.  A survey of medical image registration on graphics hardware , 2011, Comput. Methods Programs Biomed..

[13]  Barbara Cutler,et al.  Robust Adaptive 3-D Segmentation of Vessel Laminae From Fluorescence Confocal Microscope Images and Parallel GPU Implementation , 2010, IEEE Transactions on Medical Imaging.

[14]  Eric A. Hoffman,et al.  Extraction of Airways From CT (EXACT'09) , 2012, IEEE Transactions on Medical Imaging.

[15]  Aly A. Farag,et al.  On the Extraction of Curve Skeletons using Gradient Vector Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[16]  Attila Kuba,et al.  A Parallel 3D 12-Subiteration Thinning Algorithm , 1999, Graph. Model. Image Process..

[17]  Mohan M. Trivedi,et al.  Particle filtering with rendered models: A two pass approach to multi-object 3D tracking with the GPU , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[18]  John Paul Walters,et al.  Evaluating the use of GPUs in liver image segmentation and HMMER database searches , 2009, 2009 IEEE International Symposium on Parallel & Distributed Processing.

[19]  Jim Jeffers Intel® Xeon Phi™ Coprocessors , 2013 .

[20]  Yang-Lang Chang,et al.  Accelerating the Kalman Filter on a GPU , 2011, 2011 IEEE 17th International Conference on Parallel and Distributed Systems.

[21]  Stephen R. Aylward,et al.  Initialization, noise, singularities, and scale in height ridge traversal for tubular object centerline extraction , 2002, IEEE Transactions on Medical Imaging.

[22]  Takayuki Okatani,et al.  Application of the mean field methods to MRF optimization in computer vision , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Bram van Ginneken,et al.  Computer analysis of computed tomography scans of the lung: a survey , 2006, IEEE Transactions on Medical Imaging.

[24]  Jian Sun,et al.  Parallel graph-cuts by adaptive bottom-up merging , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[25]  William E. Lorensen,et al.  Marching cubes: a high resolution 3D surface construction algorithm , 1996 .

[26]  Jerry L Prince,et al.  Current methods in medical image segmentation. , 2000, Annual review of biomedical engineering.

[27]  J. Besag On the Statistical Analysis of Dirty Pictures , 1986 .

[28]  May D. Wang,et al.  High speed processing of biomedical images using programmable GPU , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[29]  Mircea Andrecut,et al.  Parallel GPU Implementation of Iterative PCA Algorithms , 2008, J. Comput. Biol..

[30]  G. Amdhal,et al.  Validity of the single processor approach to achieving large scale computing capabilities , 1967, AFIPS '67 (Spring).

[31]  Victor Podlozhnyuk,et al.  Image Convolution with CUDA , 2007 .

[32]  Jun Zhang,et al.  The Mean Field Theory In EM Procedures For Markov Random Fields , 1991, Proceedings of the Seventh Workshop on Multidimensional Signal Processing.

[33]  Fredrik Kahl,et al.  Parallel and distributed graph cuts by dual decomposition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[34]  James A. Sethian,et al.  Level Set Methods and Fast Marching Methods , 1999 .

[35]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[36]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[37]  A. F. Adams,et al.  The Survey , 2021, Dyslexia in Higher Education.

[38]  Rolf Adams,et al.  Seeded Region Growing , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Luc Van Gool,et al.  GPU-Based Foreground-Background Segmentation using an Extended Colinearity Criterion , 2005 .

[40]  Timothy F. Cootes,et al.  Active Appearance Models , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[41]  P. Sadayappan,et al.  High-performance code generation for stencil computations on GPU architectures , 2012, ICS '12.

[42]  Jörgen Ahlberg,et al.  An Active Model for Facial Feature Tracking , 2002, EURASIP J. Adv. Signal Process..

[43]  Gérard G. Medioni,et al.  Mutual information computation and maximization using GPU , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[44]  Hans-Peter Seidel,et al.  On-the-fly Point Clouds through Histogram Pyramids , 2006 .

[45]  Markus Hadwiger,et al.  Scalable and Interactive Segmentation and Visualization of Neural Processes in EM Datasets , 2009, IEEE Transactions on Visualization and Computer Graphics.

[46]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[47]  M. Akca GENERALIZED PROCRUSTES ANALYSIS AND ITS APPLICATIONS IN PHOTOGRAMMETRY , 2003 .

[48]  Marcel van Herk A fast algorithm for local minimum and maximum filters on rectangular and octagonal kernels , 1992, Pattern Recognit. Lett..

[49]  Joseph Ross Mitchell,et al.  Sketch-based volumetric seeded region growing , 2006, SBM'06.

[50]  Alois Knoll,et al.  A GPU-accelerated particle filter with pixel-level likelihood , 2008, VMV.

[51]  Martin Styner,et al.  Comparison and Evaluation of Methods for Liver Segmentation From CT Datasets , 2009, IEEE Transactions on Medical Imaging.

[52]  J. Serra Introduction to mathematical morphology , 1986 .

[53]  Luc Vincent,et al.  Watersheds in Digital Spaces: An Efficient Algorithm Based on Immersion Simulations , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[54]  Rodney A. Kennedy,et al.  A Survey of Medical Image Registration on Multicore and the GPU , 2010, IEEE Signal Processing Magazine.

[55]  Jianya Gong,et al.  GPU-accelerated MRF segmentation algorithm for SAR images , 2012, Comput. Geosci..

[56]  Frank Lindseth,et al.  GPU-Based Airway Segmentation and Centerline Extraction for Image Guided Bronchoscopy , 2012 .

[57]  John D. Owens,et al.  GPU Computing , 2008, Proceedings of the IEEE.

[58]  Max Mignotte,et al.  Markovian segmentation and parameter estimation on graphics hardware , 2006, J. Electronic Imaging.

[59]  Andreas Nüchter,et al.  GPU-Accelerated Nearest Neighbor Search for 3D Registration , 2009, ICVS.

[60]  Joseph Ross Mitchell,et al.  A work-efficient GPU algorithm for level set segmentation , 2010, HPG '10.

[61]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[62]  M.D. McCool,et al.  Scalable Programming Models for Massively Multicore Processors , 2008, Proceedings of the IEEE.

[63]  Ross T. Whitaker,et al.  A Streaming Narrow-Band Algorithm: Interactive Computation and Visualization of Level Sets , 2004, IEEE Trans. Vis. Comput. Graph..

[64]  Claude Kauffmann,et al.  Cellular automaton for ultra-fast watershed transform on GPU , 2008, 2008 19th International Conference on Pattern Recognition.

[65]  Horst Bischof,et al.  Airway Tree Reconstruction Based on Tube Detection , 2009, MICCAI 2009.

[66]  Joachim Denzler,et al.  GPU-Based Volume Segmentation , 2005 .

[67]  David W. Capson,et al.  A Framework for 3D Model-Based Visual Tracking Using a GPU-Accelerated Particle Filter , 2012, IEEE Transactions on Visualization and Computer Graphics.

[68]  H. D. Cheng,et al.  Medical image processing , 2005, Inf. Sci..

[69]  Lin Shi,et al.  A survey of GPU-based medical image computing techniques. , 2012, Quantitative imaging in medicine and surgery.

[70]  Tao Li,et al.  A robust parametric active contour based on fourier descriptors , 2011, 2011 18th IEEE International Conference on Image Processing.

[71]  Gabor Fichtinger,et al.  GPU accelerated registration of a statistical shape model of the lumbar spine to 3D ultrasound images , 2011, Medical Imaging.

[72]  Timothy F. Cootes,et al.  A minimum description length approach to statistical shape modeling , 2002, IEEE Transactions on Medical Imaging.

[73]  P. J. Narayanan,et al.  CUDA cuts: Fast graph cuts on the GPU , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[74]  Ulf Assarsson,et al.  Efficient stream compaction on wide SIMD many-core architectures , 2009, High Performance Graphics.

[75]  Julio C. Rolon,et al.  Medical image segmentation with deformable models on graphics processing units , 2013, The Journal of Supercomputing.

[76]  J. Anderson,et al.  Assessing fabric stain release with a GPU implementation of statistical snakes , 2009, Electronic Imaging.

[77]  Roberto de Alencar Lotufo,et al.  A Proposal for a Parallel Watershed Transform Algorithm for Real-Time Segmentation , 2009 .

[78]  Martin Rumpf,et al.  Level set segmentation in graphics hardware , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[79]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[80]  Hans-Peter Meinzer,et al.  Statistical shape models for 3D medical image segmentation: A review , 2009, Medical Image Anal..

[81]  Anthony J. Sherbondy,et al.  Fast volume segmentation with simultaneous visualization using programmable graphics hardware , 2003, IEEE Visualization, 2003. VIS 2003..

[82]  Isabelle Bloch,et al.  A review of 3D vessel lumen segmentation techniques: Models, features and extraction schemes , 2009, Medical Image Anal..

[83]  Nikos Paragios,et al.  Handbook of Mathematical Models in Computer Vision , 2005 .

[84]  Horst Bischof,et al.  Extracting Curve Skeletons from Gray Value Images for Virtual Endoscopy , 2008, MIAR.

[85]  Christer Sjöström,et al.  State-of-the-art report , 1997 .

[86]  Kazuhiro Otsuka,et al.  Simultaneous and fast 3D tracking of multiple faces in video by GPU-based stream processing , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[87]  P. J. Narayanan,et al.  Accelerating Large Graph Algorithms on the GPU Using CUDA , 2007, HiPC.

[88]  Junaed Sattar Snakes , Shapes and Gradient Vector Flow , 2022 .

[89]  Jos B. T. M. Roerdink,et al.  The Watershed Transform: Definitions, Algorithms and Parallelization Strategies , 2000, Fundam. Informaticae.

[90]  Michael A. Greenspan,et al.  The parallel iterative closest point algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[91]  Matthew J. Thurley,et al.  Fast Morphological Image Processing Open-Source Extensions for GPU Processing With CUDA , 2012, IEEE Journal of Selected Topics in Signal Processing.

[92]  Horst Bischof,et al.  Segmentation of Airways Based on Gradient Vector Flow , 2009, MICCAI 2009.

[93]  Olga Veksler,et al.  Graph Cuts in Vision and Graphics: Theories and Applications , 2006, Handbook of Mathematical Models in Computer Vision.

[94]  Frank Lindseth,et al.  Real-time gradient vector flow on GPUs using OpenCL , 2015, Journal of Real-Time Image Processing.

[95]  Lei Xing,et al.  GPU computing in medical physics: a review. , 2011, Medical physics.

[96]  Nicholas Ayache,et al.  Model-Based Detection of Tubular Structures in 3D Images , 2000, Comput. Vis. Image Underst..

[97]  Taku Komura,et al.  Topology matching for fully automatic similarity estimation of 3D shapes , 2001, SIGGRAPH.

[98]  Raphaël Couturier,et al.  GPU Implementation of a Region Based Algorithm for Large Images Segmentation , 2011, 2011 IEEE 11th International Conference on Computer and Information Technology.

[99]  Jesús Jiménez,et al.  Three‐dimensional thinning algorithms on graphics processing units and multicore CPUs , 2012, Concurr. Comput. Pract. Exp..

[100]  Falko Kuester,et al.  GPU-Based Active Contour Segmentation Using Gradient Vector Flow , 2006, ISVC.

[101]  Zuoyong Zheng,et al.  A Fast GVF Snake Algorithm on the GPU , 2013 .

[102]  Frank Lindseth,et al.  GPU accelerated segmentation and centerline extraction of tubular structures from medical images , 2013, International Journal of Computer Assisted Radiology and Surgery.

[103]  Max A. Viergever,et al.  Mutual-information-based registration of medical images: a survey , 2003, IEEE Transactions on Medical Imaging.

[104]  Ning Wang,et al.  Angiogram Images Enhancement Method Based on GPU , 2013 .

[105]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[106]  Ole Christian Eidheim,et al.  Real-time analysis of ultrasound images using GPU , 2005 .

[107]  Michael Werman,et al.  Computing 2-D Min, Median, and Max Filters , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[108]  Pavel Karas Efficient Computation of Morphological Greyscale Reconstruction , 2010, MEMICS.

[109]  Mohamed E. Hussein,et al.  On Implementing Graph Cuts on CUDA , 2007 .

[110]  Markku Hauta-Kasari,et al.  Real-time PCA calculation for spectral imaging (using SIMD and GP-GPU) , 2010, Journal of Real-Time Image Processing.

[111]  Alejandro F. Frangi,et al.  Muliscale Vessel Enhancement Filtering , 1998, MICCAI.

[112]  Nikos Komodakis,et al.  Markov Random Field modeling, inference & learning in computer vision & image understanding: A survey , 2013, Comput. Vis. Image Underst..

[113]  Kazuhiro Otsuka,et al.  Real-time Visual Tracker by Stream Processing , 2009, J. Signal Process. Syst..

[114]  Diego Cazorla,et al.  A GPU-based implementation of the MRF algorithm in ITK package , 2011, The Journal of Supercomputing.

[115]  Til Aach,et al.  Challenges of medical image processing , 2011, Computer Science - Research and Development.

[116]  Janito Vaqueiro Ferreira,et al.  Advances on Watershed Processing on GPU Architecture , 2011, ISMM.

[117]  Marius Erdt,et al.  Automatic Hepatic Vessel Segmentation Using Graphics Hardware , 2008, MIAR.

[118]  Anders Eklund,et al.  Medical image processing on the GPU - Past, present and future , 2013, Medical Image Anal..

[119]  Nadia Magnenat-Thalmann,et al.  A GPU framework for parallel segmentation of volumetric images using discrete deformable models , 2010, The Visual Computer.