Comparison and Evaluation of Methods for Liver Segmentation From CT Datasets

This paper presents a comparison study between 10 automatic and six interactive methods for liver segmentation from contrast-enhanced CT images. It is based on results from the "MICCAI 2007 Grand Challenge" workshop, where 16 teams evaluated their algorithms on a common database. A collection of 20 clinical images with reference segmentations was provided to train and tune algorithms in advance. Participants were also allowed to use additional proprietary training data for that purpose. All teams then had to apply their methods to 10 test datasets and submit the obtained results. Employed algorithms include statistical shape models, atlas registration, level-sets, graph-cuts and rule-based systems. All results were compared to reference segmentations five error measures that highlight different aspects of segmentation accuracy. All measures were combined according to a specific scoring system relating the obtained values to human expert variability. In general, interactive methods reached higher average scores than automatic approaches and featured a better consistency of segmentation quality. However, the best automatic methods (mainly based on statistical shape models with some additional free deformation) could compete well on the majority of test images. The study provides an insight in performance of different segmentation approaches under real-world conditions and highlights achievements and limitations of current image analysis techniques.

Martin Styner | Jacob D. Furst | Thomas Lange | Benoit M. Dawant | Brian Lennon | Fernando Bello | Horst Bischof | Hans-Peter Meinzer | Bram van Ginneken | Ivo Wolf | Joachim Hornegger | Akinobu Shimizu | Hidefumi Kobatake | Christoph Becker | Yulia Arzhaeva | Gábor Németh | Daniela Stan Raicu | Alexander Bornik | Jeongjin Lee | Eva M. van Rikxoort | Richard Kitney | Tobias Heimann | László Ruskó | Hans Lamecker | Reinhard Beichel | Erich Sorantin | Andreas Wimmer | Lars Grenacher | Volker Aurich | Mikaël Rousson | Christian Bauer | Gerd Karl Binnig | Grzegorz Soza | Ying Chi | Rui Li | Günter Schmidt | Dieter Seghers | Senhu Li | Andrés Cordova | Ruchaneewan Susomboon | Andreas Beck | Dagmar Kainmüller | Anne-Mareike Rau | Daisuke Furukawa | Márta Fidrich | Pieter Slagmolen | Jonathan M. Waite | György Bekes | Kinda Anna Saddi | Peter Cashman | B. Dawant | H. Bischof | G. Binnig | M. Rousson | Y. Arzhaeva | B. Ginneken | T. Heimann | M. Styner | E. Sorantin | J. Hornegger | H. Meinzer | I. Wolf | H. Lamecker | T. Lange | R. Kitney | J. Furst | Rui Li | G. Soza | A. Shimizu | Ying Chi | D. Raicu | K. A. Saddi | L. Ruskó | György Bekes | G. Németh | M. Fidrich | Ruchaneewan Susomboon | A. Beck | V. Aurich | Jeongjin Lee | H. Kobatake | F. Bello | G. Schmidt | R. Beichel | L. Grenacher | Christian Bauer | E. M. Rikxoort | Dagmar Kainmüller | P. Slagmolen | A. Wimmer | Senhu Li | Christoph Becker | P. Cashman | D. Seghers | D. Furukawa | A. Bornik | A. Cordova | B. Lennon | Anne-Mareike Rau | Mikaël Rousson

[1]  Konstantina S. Nikita,et al.  A computer-aided diagnostic system to characterize CT focal liver lesions: design and optimization of a neural network classifier , 2003, IEEE Transactions on Information Technology in Biomedicine.

[2]  Dieter Schmalstieg,et al.  Interactive editing of segmented volumetric datasets in a hybrid 2D/3D virtual environment , 2006, VRST '06.

[3]  Daniel Rueckert,et al.  Nonrigid registration using free-form deformations: application to breast MR images , 1999, IEEE Transactions on Medical Imaging.

[4]  R P Velthuizen,et al.  MRI segmentation: methods and applications. , 1995, Magnetic resonance imaging.

[5]  William M. Wells,et al.  Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation , 2004, IEEE Transactions on Medical Imaging.

[6]  Horst Bischof,et al.  Liver Segmentation in CT Data: A Segmentation Refinement Approach , 2007 .

[7]  Kunio Doi,et al.  Automated detection of pulmonary nodules in helical computed tomography images of the thorax , 1998, Medical Imaging.

[8]  E. Hoffman,et al.  Lung image database consortium: developing a resource for the medical imaging research community. , 2004, Radiology.

[9]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[10]  Maria Athelogou,et al.  Cognition Network Technology for a Fully Automated 3D Segmentation of Liver , 2007 .

[11]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[12]  Volker Aurich,et al.  HepaTux - A Semiautomatic Liver Segmentation System , 2007 .

[13]  Klaus D. Tönnies,et al.  Segmentation of medical images using adaptive region growing , 2001, SPIE Medical Imaging.

[14]  H. Peitgen,et al.  Computer-Assisted Operative Planning in Adult Living Donor Liver Transplantation: A New Way to Resolve the Dilemma of the Middle Hepatic Vein , 2005, World Journal of Surgery.

[15]  Guy Marchal,et al.  Multi-modality image registration by maximization of mutual information , 1996, Proceedings of the Workshop on Mathematical Methods in Biomedical Image Analysis.

[16]  Thomas Lange,et al.  Shape Constrained Automatic Segmentation of the Liver based on a Heuristic Intensity Model , 2007 .

[17]  Hans-Peter Meinzer,et al.  Computerized planning of liver surgery - an overview , 2002, Comput. Graph..

[18]  Josef Kittler,et al.  Moderating k-NN Classifiers , 2002, Pattern Analysis & Applications.

[19]  Marko Subasic,et al.  Level Set Methods and Fast Marching Methods , 2003 .

[20]  P. Jaccard,et al.  Etude comparative de la distribution florale dans une portion des Alpes et des Jura , 1901 .

[21]  Paul Suetens,et al.  Landmark based liver segmentation using local shape and local intensity models , 2007 .

[22]  Bram van Ginneken,et al.  Automatic segmentation of the liver in computed tomography scans with voxel classification and atlas matching , 2007 .

[23]  Guido Gerig,et al.  Valmet: A New Validation Tool for Assessing and Improving 3D Object Segmentation , 2001, MICCAI.

[24]  Dieter Schmalstieg,et al.  A Hybrid User Interface for Manipulation of Volumetric Medical Data , 2006, 3D User Interfaces (3DUI'06).

[25]  Fernando Bello,et al.  A Discussion on the Evaluation of A New Automatic Liver Volume Segmentation Method for Specified CT Image Datasets , 2007 .

[26]  Joachim Hornegger,et al.  Two-stage Semi-automatic Organ Segmentation Framework using Radial Basis Functions and Level Sets , 2007 .

[27]  Edward Chaney,et al.  Methods for image segmentation should be standardized and calibrated. , 2005, Medical physics.

[28]  Baba C. Vemuri,et al.  Shape Modeling with Front Propagation: A Level Set Approach , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[30]  Ross T. Whitaker,et al.  Variable-conductance, level-set curvature for image denoising , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[31]  Martin Styner,et al.  Multi-site validation of image analysis methods - Assessing intra and inter-site variability , 2002 .

[32]  W. Eric L. Grimson,et al.  A shape-based approach to the segmentation of medical imagery using level sets , 2003, IEEE Transactions on Medical Imaging.

[33]  G. McLachlan,et al.  The EM Algorithm and Extensions: Second Edition , 2008 .

[34]  Guy Marchal,et al.  Multimodality image registration by maximization of mutual information , 1997, IEEE Transactions on Medical Imaging.

[35]  Daniel Cremers,et al.  Efficient Kernel Density Estimation of Shape and Intensity Priors for Level Set Segmentation , 2005, MICCAI.

[36]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[37]  David R. Haynor,et al.  PET-CT image registration in the chest using free-form deformations , 2003, IEEE Transactions on Medical Imaging.

[38]  Richard K. Beatson,et al.  Reconstruction and representation of 3D objects with radial basis functions , 2001, SIGGRAPH.

[39]  Hans-Peter Meinzer,et al.  A Shape-Guided Deformable Model with Evolutionary Algorithm Initialization for 3D Soft Tissue Segmentation , 2007, IPMI.

[40]  Y. J. Zhang,et al.  A survey on evaluation methods for image segmentation , 1996, Pattern Recognit..

[41]  Joon Beom Seo,et al.  Ecient Liver Segmentation exploiting Level-Set Speed Images with 2.5D Shape Propagation , 2007 .

[42]  Hans-Paul Schwefel,et al.  Evolution and optimum seeking , 1995, Sixth-generation computer technology series.

[43]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[44]  Hans-Peter Meinzer,et al.  New methods for leak detection and contour correction in seeded region growing segmentation , 2004 .

[45]  E-Liang Chen,et al.  An automatic diagnostic system for CT liver image classification , 1998, IEEE Transactions on Biomedical Engineering.

[46]  F. Cheriet,et al.  Global-to-Local Shape Matching for Liver Segmentation in CT Imaging , 2007 .

[47]  Shigeo Abe DrEng Pattern Classification , 2001, Springer London.

[48]  Martha Elizabeth Shenton,et al.  On evaluating brain tissue classifiers without a ground truth , 2007, NeuroImage.

[49]  Hans-Peter Meinzer,et al.  A Statistical Deformable Model for the Segmentation of Liver CT Volumes , 2007 .

[50]  Guido Gerig,et al.  User-guided 3D active contour segmentation of anatomical structures: Significantly improved efficiency and reliability , 2006, NeuroImage.

[51]  T Heimann,et al.  Automatic Generation of 3D Statistical Shape Models with Optimal Landmark Distributions , 2007, Methods of Information in Medicine.

[52]  Xiaodong Wu,et al.  Simultaneous Segmentation of Multiple Closed Surfaces Using Optimal Graph Searching , 2005, IPMI.

[53]  L. Ruskó,et al.  Fully automatic liver segmentation for contrast-enhanced CT images , 2007 .

[54]  James A. Sethian,et al.  Level Set Methods and Fast Marching Methods , 1999 .

[55]  Timothy F. Cootes,et al.  Use of active shape models for locating structures in medical images , 1994, Image Vis. Comput..

[56]  Daniel P. Huttenlocher,et al.  Comparing Images Using the Hausdorff Distance , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[57]  Gareth Funka-Lea,et al.  Graph Cuts and Efficient N-D Image Segmentation , 2006, International Journal of Computer Vision.

[58]  Frederik Maes,et al.  Atlas based liver segmentation using nonrigid registration with a B-spline transformation model , 2007 .

[59]  J. Furst,et al.  A Hybrid Approach for Liver Segmentation , 2007 .

[60]  Timothy F. Cootes,et al.  The Use of Active Shape Models for Locating Structures in Medical Images , 1993, IPMI.

[61]  Max A. Viergever,et al.  Error Metrics for Quantitative Evaluation of Medical Image Segmentation , 1998, Theoretical Foundations of Computer Vision.

[62]  J. MacQueen Some methods for classification and analysis of multivariate observations , 1967 .

[63]  Keith Price,et al.  Anything you can do, I can do better (No you can't) , 1986, Comput. Vis. Graph. Image Process..

[64]  Kai Hormann,et al.  Surface Parameterization: a Tutorial and Survey , 2005, Advances in Multiresolution for Geometric Modelling.

[65]  Isabelle Bloch,et al.  Mathematical Morphology , 2007, Handbook of Spatial Logics.

[66]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[67]  Akinobu Shimizu,et al.  Automatic Liver Segmentation Method based on Maximum A Posterior Probability Estimation and Level Set Method , 2007 .

[68]  Johan Montagnat,et al.  Fully Automatic Anatomical, Pathological, and Functional Segmentation from CT Scans for Hepatic Surgery , 2000, Medical Imaging: Image Processing.

[69]  Sunil Arya,et al.  An optimal algorithm for approximate nearest neighbor searching fixed dimensions , 1998, JACM.

[70]  S. Osher,et al.  Geometric Level Set Methods in Imaging, Vision, and Graphics , 2011, Springer New York.

[71]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[72]  Akinobu Shimizu,et al.  Preliminary report of competition for liver region extraction algorithms from three-dimensional CT images , 2004, CARS.

[73]  Max A. Viergever,et al.  Efficient and reliable schemes for nonlinear diffusion filtering , 1998, IEEE Trans. Image Process..

[74]  Akinobu Shimizu,et al.  Segmentation of multiple organs in non-contrast 3D abdominal CT images , 2007, International Journal of Computer Assisted Radiology and Surgery.

[75]  Neil A. Dodgson,et al.  Advances in Multiresolution for Geometric Modelling , 2005 .

[76]  Michael I. Miller,et al.  Deformable templates using large deformation kinematics , 1996, IEEE Trans. Image Process..

[77]  Benoit M. Dawant,et al.  Semi-automatic Segmentation of the Liver and its Evaluation on the MICCAI 2007 Grand Challenge Data Set , 2007 .

[78]  Hyunjin Park,et al.  Construction of an abdominal probabilistic atlas and its application in segmentation , 2003, IEEE Transactions on Medical Imaging.

[79]  Benoit M. Dawant,et al.  Automatic 3D segmentation of the liver from abdominal CT images: a level-set approach , 2001, SPIE Medical Imaging.

[80]  Torsten Rohlfing,et al.  Quo Vadis, Atlas-Based Segmentation? , 2005 .