A label fusion method using conditional random fields with higher-order potentials: Application to hippocampal segmentation

OBJECTIVE The objective of this study is to develop a probabilistic modeling framework for segmenting structures of interest from a collection of atlases. We present a label fusion method that is based on minimizing an energy function using graph-cut techniques. METHODS AND MATERIALS We use a conditional random field (CRF) model that allows us to efficiently incorporate shape, appearance and context information. This model is characterized by a pseudo-Boolean function defined on unary, pairwise and higher-order potentials. Given a subset of registered atlases in the target image for a particular region of interest (ROI), we first derive an appearance-shape model from these registered atlases. The unary potentials combine an appearance model based on multiple features with a label prior using a weighted voting method. The pairwise terms are defined from a Finsler metric that minimizes the surface of separation between voxels whose labels are different. The higher-order potentials used in our framework are based on the robust P(n) model proposed by Kohli et al. The higher-order potentials enforce label consistency in cliques; hence, the proposed method can be viewed as an approach to integrate high-level information with images based on low-level features. To evaluate the performance and the robustness of the proposed label fusion method, we employ two available databases of T1-weighted (T1W) magnetic resonance (MR) images of human brains. We compare our approach with other label fusion methods in the automatic hippocampal segmentation from T1W-MR images. RESULTS Our label fusion method yields mean Dice coefficients of 0.829 and 0.790 for the two databases used with mean times of approximately 80 and 160s, respectively. CONCLUSIONS We introduce a new label fusion method based on a CRF model and on ROIs. The CRF model is characterized by a pseudo-Boolean function defined on unary, pairwise and higher-order potentials. The proposed Boolean function is representable by graphs. A globally optimal binary labeling is found using a st-mincut algorithm in each ROI. We show that the proposed approach is very competitive with respect to recently reported methods.

[1]  Stephen M Smith,et al.  Fast robust automated brain extraction , 2002, Human brain mapping.

[2]  Anil K. Jain,et al.  Small Sample Size Effects in Statistical Pattern Recognition: Recommendations for Practitioners , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  W. Eric L. Grimson,et al.  A Bayesian model for joint segmentation and registration , 2006, NeuroImage.

[4]  Jean-Michel Morel,et al.  A Review of Image Denoising Algorithms, with a New One , 2005, Multiscale Model. Simul..

[5]  W. Eric L. Grimson,et al.  Logarithm Odds Maps for Shape Representation , 2006, MICCAI.

[6]  Vladlen Koltun,et al.  Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials , 2011, NIPS.

[7]  Paul A. Viola,et al.  Alignment by Maximization of Mutual Information , 1997, International Journal of Computer Vision.

[8]  Antonio Criminisi,et al.  TextonBoost for Image Understanding: Multi-Class Object Recognition and Segmentation by Jointly Modeling Texture, Layout, and Context , 2007, International Journal of Computer Vision.

[9]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[10]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[11]  Daniel Cremers,et al.  Efficient Kernel Density Estimation of Shape and Intensity Priors for Level Set Segmentation , 2005, MICCAI.

[12]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  William M. Wells,et al.  Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation , 2004, IEEE Transactions on Medical Imaging.

[14]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[15]  Zhuowen Tu,et al.  Auto-context and its application to high-level vision tasks , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Xiao Han,et al.  Atlas Renormalization for Improved Brain MR Image Segmentation Across Scanner Platforms , 2007, IEEE Transactions on Medical Imaging.

[17]  Michael Unser,et al.  Optimization of mutual information for multiresolution image registration , 2000, IEEE Trans. Image Process..

[18]  Kiralee M. Hayashi,et al.  Conversion of mild cognitive impairment to Alzheimer disease predicted by hippocampal atrophy maps. , 2006, Archives of neurology.

[19]  Sandra E. Black,et al.  A direct morphometric comparison of five labeling protocols for multi-atlas driven automatic segmentation of the hippocampus in Alzheimer's disease , 2013, NeuroImage.

[20]  L. R. Dice.  Measures of the Amount of Ecologic Association Between Species , 1945 .

[21]  Liana G. Apostolova,et al.  Comparison of AdaBoost and Support Vector Machines for Detecting Alzheimer's Disease Through Automated Hippocampal Segmentation , 2010, IEEE Transactions on Medical Imaging.

[22]  Torsten Rohlfing,et al.  Quo Vadis, Atlas-Based Segmentation? , 2005 .

[23]  Daniel Rueckert,et al.  Segmentation of MR images via discriminative dictionary learning and sparse coding: Application to hippocampus labeling , 2013, NeuroImage.

[24]  Juha Koikkalainen,et al.  Fast and robust multi-atlas segmentation of brain magnetic resonance images , 2010, NeuroImage.

[25]  Vladimir Kolmogorov,et al.  What metrics can be approximated by geo-cuts, or global optimization of length/area and flux , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[26]  Dinggang Shen,et al.  Construction of multi-region-multi-reference atlases for neonatal brain MRI segmentation , 2010, NeuroImage.

[27]  Colin Studholme,et al.  An overlap invariant entropy measure of 3D medical image alignment , 1999, Pattern Recognit..

[28]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[29]  J. Bremner,et al.  MR-based in vivo hippocampal volumetrics: 1. Review of methodologies currently employed , 2005, Molecular Psychiatry.

[30]  Pushmeet Kohli,et al.  Robust Higher Order Potentials for Enforcing Label Consistency , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Daniel Rueckert,et al.  An evaluation of four automatic methods of segmenting the subcortical structures in the brain , 2009, NeuroImage.

[32]  Pushmeet Kohli,et al.  Minimizing sparse higher order energy functions of discrete variables , 2009, CVPR.

[33]  Jyrki Lötjönen,et al.  Measurement of hippocampal atrophy using 4D graph-cut segmentation: Application to ADNI , 2010, NeuroImage.

[34]  Dominique Hasboun,et al.  Anatomically constrained region deformation for the automated segmentation of the hippocampus and the amygdala: Method and validation on controls and patients with Alzheimer’s disease , 2007, NeuroImage.

[35]  J. Barnes,et al.  A comparison of methods for the automated calculation of volumes and atrophy rates in the hippocampus , 2008, NeuroImage.

[36]  Paul A. Yushkevich,et al.  Multi-Atlas Segmentation with Joint Label Fusion , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Colin Studholme,et al.  A Supervised Patch-Based Approach for Human Brain Labeling , 2011, IEEE Transactions on Medical Imaging.

[38]  William M. Wells,et al.  Medical Image Computing and Computer-Assisted Intervention — MICCAI’98 , 1998, Lecture Notes in Computer Science.

[39]  Torsten Rohlfing,et al.  Image Similarity and Tissue Overlaps as Surrogates for Image Registration Accuracy: Widely Used but Unreliable , 2012, IEEE Transactions on Medical Imaging.

[40]  Arthur W. Toga,et al.  Fusing adaptive atlas and informative features for robust 3D brain image segmentation , 2010, 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[41]  Hamid Soltanian-Zadeh,et al.  Dataset of Magnetic Resonance Images of Nonepileptic Subjects and Temporal Lobe Epilepsy Patients for Validation of Hippocampal Segmentation Techniques , 2011, Neuroinformatics.

[42]  Daniel Rueckert,et al.  Multi-atlas based segmentation of brain images: Atlas selection and its effect on accuracy , 2009, NeuroImage.

[43]  Zhuowen Tu,et al.  Fixed-Point Model For Structured Labeling , 2013, ICML.

[44]  Brian B. Avants,et al.  Integrated Graph Cuts for Brain MRI Segmentation , 2006, MICCAI.

[45]  A. Dale,et al.  Whole Brain Segmentation Automated Labeling of Neuroanatomical Structures in the Human Brain , 2002, Neuron.

[46]  Arno Klein,et al.  Evaluation of 14 nonlinear deformation algorithms applied to human brain MRI registration , 2009, NeuroImage.

[47]  Stephen M. Smith,et al.  Enhanced brain extraction improves the accuracy of brain atrophy estimation , 2008, NeuroImage.

[48]  Jitendra Malik,et al.  Representing and Recognizing the Visual Appearance of Materials using Three-dimensional Textons , 2001, International Journal of Computer Vision.

[49]  Carlos Ortiz-de-Solorzano,et al.  Combination Strategies in Multi-Atlas Image Segmentation: Application to Brain MR Data , 2009, IEEE Transactions on Medical Imaging.

[50]  D. Louis Collins,et al.  Towards accurate, automatic segmentation of the hippocampus and amygdala from MRI by augmenting ANIMAL with a template library and label fusion , 2010, NeuroImage.

[51]  Roberto Cipolla,et al.  Semantic texton forests for image categorization and segmentation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[52]  Chulhee Lee,et al.  Feature extraction based on the Bhattacharyya distance , 2003, Pattern Recognit..

[53]  D. Louis Collins,et al.  Patch-based segmentation using expert priors: Application to hippocampus and ventricle segmentation , 2011, NeuroImage.

[54]  Stephen M. Smith,et al.  A Bayesian model of shape and appearance for subcortical brain segmentation , 2011, NeuroImage.

[55]  Daniel Rueckert,et al.  Nonrigid registration using free-form deformations: application to breast MR images , 1999, IEEE Transactions on Medical Imaging.

[56]  Max A. Viergever,et al.  elastix: A Toolbox for Intensity-Based Medical Image Registration , 2010, IEEE Transactions on Medical Imaging.

[57]  Stephen M. Smith,et al.  Improved Optimization for the Robust and Accurate Linear Registration and Motion Correction of Brain Images , 2002, NeuroImage.

[58]  Mert R. Sabuncu,et al.  A Generative Model for Image Segmentation Based on Label Fusion , 2010, IEEE Transactions on Medical Imaging.

[59]  Daniel Rueckert,et al.  Automatic anatomical brain MRI segmentation combining label propagation and decision fusion , 2006, NeuroImage.

[60]  Daoqiang Zhang,et al.  A generative probability model of joint label fusion for multi-atlas based brain segmentation , 2014, Medical Image Anal..

[61]  Wiro J. Niessen,et al.  Hippocampus segmentation in MR images using atlas registration, voxel classification, and graph cuts , 2008, NeuroImage.

[62]  Josien P. W. Pluim,et al.  Evaluation of Optimization Methods for Nonrigid Medical Image Registration Using Mutual Information and B-Splines , 2007, IEEE Transactions on Image Processing.

[63]  Paul M. Thompson,et al.  Brain Anatomical Structure Segmentation by Hybrid Discriminative/Generative Models , 2008, IEEE Transactions on Medical Imaging.

[64]  Nicholas Lange,et al.  Neuron Numbers and Volume of the Amygdala in Subjects Diagnosed with Bipolar Disorder or Schizophrenia , 2007, Biological Psychiatry.

[65]  P. Thompson,et al.  Brain mapping as a tool to study neurodegeneration , 2007, Neurotherapeutics.