Inter-Method Discrepancies in Brain Volume Estimation May Drive Inconsistent Findings in Autism

Previous studies applying automatic preprocessing methods on Structural Magnetic Resonance Imaging (sMRI) report inconsistent neuroanatomical abnormalities in Autism Spectrum Disorder (ASD). In this study we investigate inter-method differences as a possible cause behind these inconsistent findings. In particular, we focus on the estimation of the following brain volumes: gray matter (GM), white matter (WM), cerebrospinal fluid (CSF), and total intra cranial volume (TIV). T1-weighted sMRIs of 417 ASD subjects and 459 typically developing controls (TDC) from the ABIDE dataset were estimated using three popular preprocessing methods: SPM, FSL, and FreeSurfer (FS). Brain volumes estimated by the three methods were correlated but had significant inter-method differences; except TIVSPM vs. TIVFS, all inter-method differences were significant. ASD vs. TDC group differences in all brain volume estimates were dependent on the method used. SPM showed that TIV, GM, and CSF volumes of ASD were larger than TDC with statistical significance, whereas FS and FSL did not show significant differences in any of the volumes; in some cases, the direction of the differences were opposite to SPM. When methods were compared with each other, they showed differential biases for autism, and several biases were larger than ASD vs. TDC differences of the respective methods. After manual inspection, we found inter-method segmentation mismatches in the cerebellum, sub-cortical structures, and inter-sulcal CSF. In addition, to validate automated TIV estimates we performed manual segmentation on a subset of subjects. Results indicate that SPM estimates are closest to manual segmentation, followed by FS while FSL estimates were significantly lower. In summary, we show that ASD vs. TDC brain volume differences are method dependent and that these inter-method discrepancies can contribute to inconsistent neuroimaging findings in general. We suggest cross-validation across methods and emphasize the need to develop better methods to increase the robustness of neuroimaging findings.

[1]  Stephen M Smith,et al.  Fast robust automated brain extraction , 2002, Human brain mapping.

[2]  S. Lawrie,et al.  Towards a neuroanatomy of autism: A systematic review and meta-analysis of structural magnetic resonance imaging studies , 2008, European Psychiatry.

[3]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[4]  Daniel P. Kennedy,et al.  The Autism Brain Imaging Data Exchange: Towards Large-Scale Evaluation of the Intrinsic Brain Architecture in Autism , 2013, Molecular Psychiatry.

[5]  Sigal Berman,et al.  Anatomical Abnormalities in Autism? , 2016, Cerebral cortex.

[6]  Jay N. Giedd,et al.  Motion Artifact in Magnetic Resonance Imaging: Implications for Automated Analysis , 2002, NeuroImage.

[7]  Stephen M. Smith,et al.  A global optimisation method for robust affine registration of brain images , 2001, Medical Image Anal..

[8]  F. Happé,et al.  Meta-analysis of gray matter abnormalities in autism spectrum disorder: should Asperger disorder be subsumed under a broader umbrella of autistic spectrum disorder? , 2011, Archives of general psychiatry.

[9]  Dinggang Shen,et al.  Unsupervised Deep Learning for Hippocampus Segmentation in 7.0 Tesla MR Images , 2013, MLMI.

[10]  Andrew M Michael,et al.  Divide and Conquer: Sub-Grouping of ASD Improves ASD Detection Based on Brain Morphometry , 2016, PloS one.

[11]  E. Courchesne,et al.  When Is the Brain Enlarged in Autism? A Meta-Analysis of All Brain Size Reports , 2005, Biological Psychiatry.

[12]  Ivo D Dinov,et al.  Structural brain atlases: design, rationale, and applications in normal and pathological cohorts. , 2012, Journal of Alzheimer's disease : JAD.

[13]  Giovanni Montana,et al.  Deep neural networks for anatomical brain segmentation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[14]  D Mataix-Cols,et al.  Voxel-based meta-analysis of regional white-matter volume differences in autism spectrum disorder versus healthy controls , 2010, Psychological Medicine.

[15]  Joseph T. Chang,et al.  Early generalized overgrowth in autism spectrum disorder: prevalence rates, gender effects, and clinical outcomes. , 2014, Journal of the American Academy of Child and Adolescent Psychiatry.

[16]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[17]  N. Muhlert,et al.  Failed replications, contributing factors and careful interpretations: Commentary on Boekel et al., 2015 , 2016, Cortex.

[18]  Guido Gerig,et al.  User-guided 3D active contour segmentation of anatomical structures: Significantly improved efficiency and reliability , 2006, NeuroImage.

[19]  Neil D. Woodward,et al.  Brain structure in autism: a voxel-based morphometry analysis of the Autism Brain Imaging Database Exchange (ABIDE) , 2016, Brain Imaging and Behavior.

[20]  Michael P Milham,et al.  Multicenter mapping of structural network alterations in autism , 2015, Human brain mapping.

[21]  Guido Gerig,et al.  Multisite validation of image analysis methods: assessing intra- and intersite variability , 2002, SPIE Medical Imaging.

[22]  Brian A. Nosek,et al.  Power failure: why small sample size undermines the reliability of neuroscience , 2013, Nature Reviews Neuroscience.

[23]  D. Amaral,et al.  Neuroanatomy of autism , 2008, Trends in Neurosciences.

[24]  Nasser Kehtarnavaz,et al.  Comparison of tissue segmentation algorithms in neuroimage analysis software tools , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[25]  F. E. Satterthwaite.  An approximate distribution of estimates of variance components. , 1946, Biometrics.

[26]  A. Dale,et al.  High‐resolution intersubject averaging and a coordinate system for the cortical surface , 1999, Human brain mapping.

[27]  Scott D. Brown,et al.  A purely confirmatory replication study of structural brain-behavior correlations , 2015, Cortex.

[28]  B A Ardekani,et al.  Corpus Callosum Area and Brain Volume in Autism Spectrum Disorder: Quantitative Analysis of Structural MRI from the ABIDE Database , 2015, Journal of autism and developmental disorders.

[29]  P. Brockhoff,et al.  lmerTest: Tests for random and fixed effects for linear mixed effect models (lmer objects of lme4 package) , 2014 .

[30]  S. Spence,et al.  Compared to What? Early Brain Overgrowth in Autism and the Perils of Population Norms , 2013, Biological Psychiatry.

[31]  J. Brian,et al.  Early head growth in infants at risk of autism: a baby siblings research consortium study. , 2014, Journal of the American Academy of Child and Adolescent Psychiatry.

[32]  L. Schad,et al.  Comparison of automated brain segmentation using a brain phantom and patients with early Alzheimer's dementia or mild cognitive impairment , 2015, Psychiatry Research: Neuroimaging.

[33]  Rong Xu,et al.  Segmentation of Brain MRI , 2012 .

[34]  Satterthwaite Fe.  An approximate distribution of estimates of variance components. , 1946 .

[35]  Martin Styner,et al.  Multi-site validation of image analysis methods - Assessing intra and inter-site variability , 2002 .

[36]  Josephine Barnes,et al.  Estimation of total intracranial volume; a comparison of methods , 2011, Alzheimer's & Dementia.

[37]  Anders M. Dale,et al.  Cortical Surface-Based Analysis I. Segmentation and Surface Reconstruction , 1999, NeuroImage.

[38]  Anders M. Dale,et al.  Prospective motion correction of high-resolution magnetic resonance imaging data in children , 2010, NeuroImage.

[39]  D. Louis Collins,et al.  Application of Information Technology: A Four-Dimensional Probabilistic Atlas of the Human Brain , 2001, J. Am. Medical Informatics Assoc..

[40]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[41]  G. Yue,et al.  Do preprocessing algorithms and statistical models influence voxel‐based morphometry (VBM) results in amyotrophic lateral sclerosis patients? A systematic comparison of popular VBM analytical methods , 2014, Journal of magnetic resonance imaging : JMRI.

[42]  W. Tseng,et al.  Regional brain volume differences between males with and without autism spectrum disorder are highly age-dependent , 2015, Molecular Autism.

[43]  Matthew Lai,et al.  Deep Learning for Medical Image Segmentation , 2015, Deep Learning Applications in Medical Imaging.

[44]  J Mazziotta,et al.  A probabilistic atlas and reference system for the human brain: International Consortium for Brain Mapping (ICBM). , 2001, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[45]  Arthur W. Toga,et al.  A Probabilistic Atlas of the Human Brain: Theory and Rationale for Its Development The International Consortium for Brain Mapping (ICBM) , 1995, NeuroImage.

[46]  R. Tubbs,et al.  Neuroanatomical variation in autism spectrum disorder: A comprehensive review , 2016, Clinical anatomy.

[47]  Ruth A. Carper,et al.  Unusual brain growth patterns in early life in patients with autistic disorder , 2001, Neurology.

[48]  T. Vangberg,et al.  How Does the Accuracy of Intracranial Volume Measurements Affect Normalized Brain Volumes? Sample Size Estimates Based on 966 Subjects from the HUNT MRI Cohort , 2015, American Journal of Neuroradiology.

[49]  Yun Jiao,et al.  Structural MRI in Autism Spectrum Disorder , 2011, Pediatric Research.

[50]  Daniel Rueckert,et al.  A comparison of the tissue classification and the segmentation propagation techniques in MRI brain image segmentation , 2005, SPIE Medical Imaging.

[51]  Ron Mengelers,et al.  The Effects of FreeSurfer Version, Workstation Type, and Macintosh Operating System Version on Anatomical Volume and Cortical Thickness Measurements , 2012, PloS one.

[52]  Camille Breuil,et al.  Detectability of brain structure abnormalities related to autism through MRI-derived measures from multiple scanners , 2014, IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI).

[53]  Karl J. Friston,et al.  Unified segmentation , 2005, NeuroImage.

[54]  Alan C. Evans,et al.  Anatomical mapping of functional activation in stereotactic coordinate space , 1992, NeuroImage.

[55]  Tilo Kircher,et al.  Accuracy and Reliability of Automated Gray Matter Segmentation Pathways on Real and Simulated Structural Magnetic Resonance Images of the Human Brain , 2012, PloS one.

[56]  D. Collins,et al.  Automatic 3D Intersubject Registration of MR Volumetric Data in Standardized Talairach Space , 1994, Journal of computer assisted tomography.

[57]  Arthur W. Toga,et al.  The construction of a Chinese MRI brain atlas: A morphometric comparison study between Chinese and Caucasian cohorts , 2010, NeuroImage.

[58]  E. Courchesne,et al.  Unusual brain growth patterns in early life in patients with autistic disorder: An MRI study , 2001, Neurology.

[59]  Stephen M. Smith,et al.  Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm , 2001, IEEE Transactions on Medical Imaging.

[60]  Eric Courchesne,et al.  Cerebral Lobes in Autism: Early Hyperplasia and Abnormal Age Effects , 2002, NeuroImage.

[61]  Stephen M. Smith,et al.  Accurate, Robust, and Automated Longitudinal and Cross-Sectional Brain Change Analysis , 2002, NeuroImage.

[62]  E. Courchesne,et al.  Brain growth across the life span in autism: Age-specific changes in anatomical pathology , 2011, Brain Research.

[63]  Frederik Maes,et al.  Assessing age-related gray matter decline with voxel-based morphometry depends significantly on segmentation and normalization procedures , 2014, Front. Aging Neurosci..

[64]  Lars Johansson,et al.  Intracranial volume estimated with commonly used methods could introduce bias in studies including brain volume measurements , 2013, NeuroImage.

[65]  Abraham Z. Snyder,et al.  A unified approach for morphometric and functional data analysis in young, old, and demented adults using automated atlas-based head size normalization: reliability and validation against manual measurement of total intracranial volume , 2004, NeuroImage.