LINKS: Learning-based multi-source IntegratioN frameworK for Segmentation of infant brain images

Segmentation of infant brain MR images is challenging due to insufficient image quality, severe partial volume effect, and the ongoing maturation and myelination processes. In particular, the image contrast inverts around 6–8 months of age, and the white and gray matter tissues are isointense in both T1- and T2-weighted MR images and thus exhibit the extremely low tissue contrast, which poses the significant challenges for automated segmentation. Most previous studies used multi-atlas label fusion strategy, which has the limitation of equally treating the available multi-modality images and is often computationally expensive. In this paper, we propose a novel learning-based multi-source integration framework for infant brain image segmentation. Specifically, we employ the random forest technique to effectively integrate features from multi-source images together for tissue segmentation. The multi-source images include initially only the multi-modality (T1, T2 and FA) images and later also the iteratively estimated and refined tissue probability maps of gray matter, white matter, and cerebrospinal fluid. Experimental results on 119 infant subjects and MICCAI challenges show that the proposed method achieves better performance than other state-of-the-art automated segmentation methods, with significantly reduction of running time from hours to 5 minutes.

[1]  Simon K. Warfield,et al.  Segmentation of newborn brain MRI , 2006, 3rd IEEE International Symposium on Biomedical Imaging: Nano to Macro, 2006..

[2]  Guido Gerig,et al.  Unbiased diffeomorphic atlas construction for computational anatomy , 2004, NeuroImage.

[3]  Zhuowen Tu,et al.  Auto-Context and Its Application to High-Level Vision Tasks and 3D Brain Image Segmentation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Dinggang Shen,et al.  Neonatal brain image segmentation in longitudinal MRI studies , 2010, NeuroImage.

[5]  Yihong Gong,et al.  Linear spatial pyramid matching using sparse coding for image classification , 2009, CVPR.

[6]  Daniel Rueckert,et al.  Automatic segmentation and reconstruction of the cortex from neonatal MRI , 2007, NeuroImage.

[7]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[8]  Alan C. Evans,et al.  A nonparametric method for automatic correction of intensity nonuniformity in MRI data , 1998, IEEE Transactions on Medical Imaging.

[9]  Dinggang Shen,et al.  journal homepage: www.elsevier.com/locate/ynimg , 2022 .

[10]  Yaozong Gao,et al.  Integration of sparse multi-modality representation and anatomical constraint for isointense infant brain MR image segmentation , 2014, NeuroImage.

[11]  John H. Gilmore,et al.  Correction: Family Poverty Affects the Rate of Human Infant Brain Growth , 2015, PloS one.

[12]  Daniel Rueckert,et al.  Multi-atlas based segmentation of brain images: Atlas selection and its effect on accuracy , 2009, NeuroImage.

[13]  Jean Ponce,et al.  Task-Driven Dictionary Learning , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  J. Mangin,et al.  Atlas-Free Surface Reconstruction of the Cortical Grey-White Interface in Infants , 2011, PloS one.

[15]  Ron Kikinis,et al.  Adaptive, template moderated, spatially varying statistical classification , 2000, Medical Image Anal..

[16]  Dinggang Shen,et al.  A computational growth model for measuring dynamic cortical development in the first year of life. , 2012, Cerebral cortex.

[17]  Yaozong Gao,et al.  Context-Aware Anatomical Landmark Detection: Application to Deformable Model Initialization in Prostate CT Images , 2014, MLMI.

[18]  Yaozong Gao,et al.  Hierarchical Lung Field Segmentation With Joint Shape and Appearance Sparse Learning , 2014, IEEE Transactions on Medical Imaging.

[19]  Dinggang Shen,et al.  CENTS: Cortical enhanced neonatal tissue segmentation , 2011, Human brain mapping.

[20]  J. Gilmore,et al.  Mapping Longitudinal Development of Local Cortical Gyrification in Infants from Birth to 2 Years of Age , 2014, The Journal of Neuroscience.

[21]  J. Gilmore,et al.  Mapping region-specific longitudinal cortical surface expansion from birth to 2 years of age. , 2013, Cerebral cortex.

[22]  Shu Liao,et al.  Automated Segmentation of CBCT Image Using Spiral CT Atlases and Convex Optimization , 2013, MICCAI.

[23]  Laura Gui,et al.  Morphology-driven automatic segmentation of MR images of the neonatal brain , 2012, Medical Image Anal..

[24]  Guillermo Sapiro,et al.  Sparse Representation for Computer Vision and Pattern Recognition , 2010, Proceedings of the IEEE.

[25]  Juha Koikkalainen,et al.  Fast and robust multi-atlas segmentation of brain magnetic resonance images , 2010, NeuroImage.

[26]  Andrew Zisserman,et al.  Image Classification using Random Forests and Ferns , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[27]  Dimitris N. Metaxas,et al.  Deformable segmentation via sparse representation and dictionary learning , 2012, Medical Image Anal..

[28]  Dinggang Shen,et al.  iBEAT: A Toolbox for Infant Brain Magnetic Resonance Image Processing , 2012, Neuroinformatics.

[29]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Jay N. Giedd,et al.  Motion Artifact in Magnetic Resonance Imaging: Implications for Automated Analysis , 2002, NeuroImage.

[31]  Esa Alhoniemi,et al.  Gaussian mixture model-based segmentation of MR images taken from premature infant brains , 2009, Journal of Neuroscience Methods.

[32]  Dinggang Shen,et al.  Longitudinal development of cortical thickness, folding, and fiber density networks in the first 2 years of life , 2014, Human brain mapping.

[33]  Daoqiang Zhang,et al.  Machine learning in medical imaging. , 2015, Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society.

[34]  Daniel Rueckert,et al.  A Probabilistic Patch-Based Label Fusion Model for Multi-Atlas Segmentation With Registration Refinement: Application to Cardiac MR Images , 2013, IEEE Transactions on Medical Imaging.

[35]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[36]  Dinggang Shen,et al.  Mapping longitudinal hemispheric structural asymmetries of the human cerebral cortex from birth to 2 years of age. , 2014, Cerebral cortex.

[37]  Dinggang Shen,et al.  Multi-atlas Based Simultaneous Labeling of Longitudinal Dynamic Cortical Surfaces in Infants , 2013, MICCAI.

[38]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Christos Davatzikos,et al.  Spatiotemporal maturation patterns of murine brain quantified by diffusion tensor MRI and deformation-based morphometry , 2005, Proc. Natl. Acad. Sci. USA.

[40]  J. Gilmore,et al.  Infant Brain Atlases from Neonates to 1- and 2-Year-Olds , 2011, PloS one.

[41]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[42]  D. Louis Collins,et al.  Optimized PatchMatch for Near Real Time and Accurate Label Fusion , 2014, MICCAI.

[43]  Dinggang Shen,et al.  4D Multi-Modality Tissue Segmentation of Serial Infant Images , 2012, PloS one.

[44]  Dimitris N. Metaxas,et al.  Entangled Decision Forests and Their Application for Semantic Segmentation of CT Images , 2011, IPMI.

[45]  Dinggang Shen,et al.  Construction of multi-region-multi-reference atlases for neonatal brain MRI segmentation , 2010, NeuroImage.

[46]  Xiao Han Learning-Boosted Label Fusion for Multi-atlas Auto-Segmentation , 2013, MLMI.

[47]  Guido Gerig,et al.  User-guided 3D active contour segmentation of anatomical structures: Significantly improved efficiency and reliability , 2006, NeuroImage.

[48]  Junzhou Huang,et al.  Towards robust and effective shape modeling: Sparse shape composition , 2012, Medical Image Anal..

[49]  Bram van Ginneken,et al.  Segmentation of the posterior ribs in chest radiographs using iterated contextual pixel classification , 2006, IEEE Transactions on Medical Imaging.

[50]  Dinggang Shen,et al.  Measuring the dynamic longitudinal cortex development in infants by reconstruction of temporally consistent cortical surfaces , 2014, NeuroImage.

[51]  Antonio Criminisi,et al.  Decision Forests with Long-Range Spatial Context for Organ Localization in CT Volumes , 2009 .

[52]  M. Styner,et al.  Longitudinal development of cortical and subcortical gray matter from birth to 2 years. , 2012, Cerebral cortex.

[53]  Torsten Rohlfing,et al.  Performance-based classifier combination in atlas-based image segmentation using expectation-maximization parameter estimation , 2004, IEEE Transactions on Medical Imaging.

[54]  Dinggang Shen,et al.  Cortical thickness and surface area in neonates at high risk for schizophrenia , 2014, Brain Structure and Function.

[55]  Yaozong Gao,et al.  Learning Distance Transform for Boundary Detection and Deformable Segmentation in CT Prostate Images , 2014, MLMI.

[56]  Ben Glocker,et al.  Decision Forests for Tissue-Specific Segmentation of High-Grade Gliomas in Multi-channel MR , 2012, MICCAI.

[57]  Yaozong Gao,et al.  Segmentation of neonatal brain MR images using patch-driven level sets , 2014, NeuroImage.

[58]  J. Gilmore,et al.  Dynamic Development of Regional Cortical Thickness and Surface Area in Early Childhood. , 2015, Cerebral cortex.

[59]  Irfan A. Essa,et al.  Tree-based Classifiers for Bilayer Video Segmentation , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[60]  Roberto Cipolla,et al.  Semantic texton forests for image categorization and segmentation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[61]  Daniel Rueckert,et al.  Segmentation of MR images via discriminative dictionary learning and sparse coding: Application to hippocampus labeling , 2013, NeuroImage.

[62]  Dinggang Shen,et al.  Constructing 4D Infant Cortical Surface Atlases Based on Dynamic Developmental Trajectories of the Cortex , 2014, MICCAI.

[63]  Dinggang Shen,et al.  Development Trends of White Matter Connectivity in the First Years of Life , 2011, PloS one.

[64]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[65]  Ben Glocker,et al.  Multi-Atlas Label Propagation with Atlas Encoding by Randomized Forests , 2013 .

[66]  Daniel Rueckert,et al.  Automatic anatomical brain MRI segmentation combining label propagation and decision fusion , 2006, NeuroImage.

[67]  Colin Studholme,et al.  A Supervised Patch-Based Approach for Human Brain Labeling , 2011, IEEE Transactions on Medical Imaging.

[68]  Yihong Gong,et al.  Locality-constrained Linear Coding for image classification , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[69]  Arno Klein,et al.  A reproducible evaluation of ANTs similarity metric performance in brain image registration , 2011, NeuroImage.

[70]  Petronella Anbeek,et al.  Probabilistic Brain Tissue Segmentation in Neonatal Magnetic Resonance Imaging , 2008, Pediatric Research.

[71]  Alan C. Evans,et al.  A fully automatic and robust brain MRI tissue classification method , 2003, Medical Image Anal..

[72]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[73]  Ben Glocker,et al.  Atlas Encoding by Randomized Forests for Efficient Label Propagation , 2013, MICCAI.

[74]  John H. Gilmore,et al.  Automatic segmentation of MR images of the developing newborn brain , 2005, Medical Image Anal..

[75]  J. Gilmore,et al.  Longitudinally guided level sets for consistent tissue segmentation of neonates , 2013, Human brain mapping.

[76]  Paul A. Yushkevich,et al.  Multi-Atlas Segmentation with Joint Label Fusion , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[77]  Antonio Criminisi,et al.  Decision Forests: A Unified Framework for Classification, Regression, Density Estimation, Manifold Learning and Semi-Supervised Learning , 2012, Found. Trends Comput. Graph. Vis..

[78]  Dinggang Shen,et al.  Automatic segmentation of neonatal images using convex optimization and coupled level sets , 2011, NeuroImage.

[79]  Daniel Rueckert,et al.  A dynamic 4D probabilistic atlas of the developing brain , 2011, NeuroImage.

[80]  D. Louis Collins,et al.  Patch-based segmentation using expert priors: Application to hippocampus and ventricle segmentation , 2011, NeuroImage.

[81]  Max A. Viergever,et al.  Automatic neonatal brain tissue segmentation with MRI , 2013, Medical Imaging.

[82]  Simon K. Warfield,et al.  Automatic segmentation of newborn brain MRI , 2009, NeuroImage.

[83]  William M. Wells,et al.  Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation , 2004, IEEE Transactions on Medical Imaging.

[84]  Ben Glocker,et al.  Encoding atlases by randomized classification forests for efficient multi-atlas label propagation , 2014, Medical Image Anal..

[85]  Hong Cheng,et al.  Sparsity induced similarity measure for label propagation , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[86]  Suyash P. Awate,et al.  Clinical Neonatal Brain MRI Segmentation Using Adaptive Nonparametric Data Models and Intensity-Based Markov Priors , 2007, MICCAI.

[87]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[88]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[89]  Shu Liao,et al.  Prostate Segmentation by Sparse Representation Based Classification , 2012, MICCAI.