Evaluating Segmentation Error without Ground Truth

The automatic delineation of the boundaries of organs and other anatomical structures is a key component of many medical image processing systems. In this paper we present a generic learning approach based on a novel space of segmentation features, which can be trained to predict the overlap error and Dice coefficient of an arbitrary organ segmentation without knowing the ground truth delineation. We show the regressor to be much stronger a predictor of these error metrics than the responses of probabilistic boosting classifiers trained on the segmentation boundary. The presented approach not only allows us to build reliable confidence measures and fidelity checks, but also to rank several segmentation hypotheses against each other during online usage of the segmentation algorithm in clinical practice.

[1]  Marie-Pierre Jolly,et al.  Interactive Organ Segmentation Using Graph Cuts , 2000, MICCAI.

[2]  B. Ginneken,et al.  3D Segmentation in the Clinic: A Grand Challenge , 2007 .

[3]  Branislav Jaramaz,et al.  Medical Image Computing and Computer-Assisted Intervention – MICCAI 2000 , 2000, Lecture Notes in Computer Science.

[4]  Timo Kohlberger,et al.  Automatic Multi-organ Segmentation Using Learning-Based Segmentation and Level Set Optimization , 2011, MICCAI.

[5]  S. Sathiya Keerthi,et al.  Improvements to the SMO algorithm for SVM regression , 2000, IEEE Trans. Neural Networks Learn. Syst..

[6]  James A. Sethian,et al.  Level Set Methods and Fast Marching Methods , 1999 .

[7]  Zhuowen Tu,et al.  Probabilistic boosting-tree: learning discriminative models for classification, recognition, and clustering , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[8]  Martin Styner,et al.  Comparison and Evaluation of Methods for Liver Segmentation From CT Datasets , 2009, IEEE Transactions on Medical Imaging.

[9]  Nicholas Ayache,et al.  Medical Image Computing and Computer-Assisted Intervention - MICCAI 2007, 10th International Conference, Brisbane, Australia, October 29 - November 2, 2007, Proceedings, Part I , 2007, MICCAI.

[10]  Dorin Comaniciu,et al.  Four-Chamber Heart Modeling and Automatic Segmentation for 3-D Cardiac CT Volumes Using Marginal Space Learning and Steerable Features , 2008, IEEE Transactions on Medical Imaging.

[11]  D. Mumford,et al.  Optimal approximations by piecewise smooth functions and associated variational problems , 1989 .

[12]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[13]  James A. Sethian,et al.  Level Set Methods and Fast Marching Methods/ J. A. Sethian , 1999 .

[14]  Mark Jenkinson,et al.  Non-local Shape Descriptor: A New Similarity Metric for Deformable Multi-modal Registration , 2011, MICCAI.

[15]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Guillermo Sapiro,et al.  Minimal Surfaces Based Object Segmentation , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Lei Zhu,et al.  On Simulating Subjective Evaluation Using Combined Objective Metrics for Validation of 3D Tumor Segmentation , 2007, MICCAI.

[18]  Leo Grady,et al.  Automating image segmentation verification and validation by learning test oracles , 2011, Inf. Softw. Technol..

[19]  Dorin Comaniciu,et al.  Hierarchical, learning-based automatic liver segmentation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Leo Grady,et al.  Isoperimetric graph partitioning for image segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Leo Grady,et al.  Random Walks for Image Segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Andrew Blake,et al.  Visual Reconstruction , 1987, Deep Learning for EEG-Based Brain–Computer Interfaces.

[23]  Guillermo Sapiro,et al.  Geodesic Active Contours , 1995, International Journal of Computer Vision.

[24]  J. A. Sethian,et al.  Fast Marching Methods , 1999, SIAM Rev..

[25]  Leo Grady,et al.  Fast global optimization of curvature , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.