Permutation and parametric tests for effect sizes in voxel-based morphometry of gray matter volume in brain structural MRI.

Permutation testing has been widely implemented in voxel-based morphometry (VBM) tools. However, this type of non-parametric inference has yet to be thoroughly compared with traditional parametric inference in VBM studies of brain structure. Here we compare both types of inference and investigate what influence the number of permutations in permutation testing has on results in an exemplar study of how gray matter proportion changes with age in a group of working age adults. High resolution T1-weighted volume scans were acquired from 80 healthy adults aged 25-64years. Using a validated VBM procedure and voxel-based permutation testing for Pearson product-moment coefficient, the effect sizes of changes in gray matter proportion with age were assessed using traditional parametric and permutation testing inference with 100, 500, 1000, 5000, 10000 and 20000 permutations. The statistical significance was set at P<0.05 and false discovery rate (FDR) was used to correct for multiple comparisons. Clusters of voxels with statistically significant (PFDR<0.05) declines in gray matter proportion with age identified with permutation testing inference (N≈6000) were approximately twice the size of those identified with parametric inference (N=3221voxels). Permutation testing with 10000 (N=6251voxels) and 20000 (N=6233voxels) permutations produced clusters that were generally consistent with each other. However, with 1000 permutations there were approximately 20% more statistically significant voxels (N=7117voxels) than with ≥10000 permutations. Permutation testing inference may provide a more sensitive method than traditional parametric inference for identifying age-related differences in gray matter proportion. Based on the results reported here, at least 10000 permutations should be used in future univariate VBM studies investigating age related changes in gray matter to avoid potential false findings. Additional studies using permutation testing in large imaging databanks are required to address the impact of model complexity, multivariate analysis, number of observations, sampling bias and data quality on the accuracy with which subtle differences in brain structure associated with normal aging can be identified.

[1]  Thomas E. Nichols,et al.  Rank-order versus mean based statistics for neuroimaging , 2007, NeuroImage.

[2]  R. McCarley,et al.  A review of MRI findings in schizophrenia , 2001, Schizophrenia Research.

[3]  Brian B. Avants,et al.  Symmetric diffeomorphic image registration with cross-correlation: Evaluating automated labeling of elderly and neurodegenerative brain , 2008, Medical Image Anal..

[4]  Jacob Cohen.  Statistical Power Analysis , 1992 .

[5]  Valérie Ventura,et al.  Bootstrap Tests of Hypotheses , 2010 .

[6]  Donald Fraser,et al.  Randomization Tests for a Multivariate Two-Sample Problem , 1958 .

[7]  L. Jäncke,et al.  Brain structural trajectories over the adult lifespan , 2012, Human brain mapping.

[8]  Karl J. Friston,et al.  Voxel-Based Morphometry—The Methods , 2000, NeuroImage.

[9]  B. Bogerts,et al.  Regional specificity of sex effects on subcortical volumes across the lifespan in healthy aging , 2014, Human brain mapping.

[10]  P. Good.  Permutation, Parametric, and Bootstrap Tests of Hypotheses , 2005 .

[11]  Karl J. Friston,et al.  A Voxel-Based Morphometric Study of Ageing in 465 Normal Adult Human Brains , 2001, NeuroImage.

[12]  J. Babb,et al.  Age-related total gray matter and white matter changes in normal adult brain. Part I: volumetric MR imaging analysis. , 2002, AJNR. American journal of neuroradiology.

[13]  John Suckling,et al.  Global, voxel, and cluster tests, by theory and permutation, for a difference between two groups of structural MR images of the brain , 1999, IEEE Transactions on Medical Imaging.

[14]  B. Qiu,et al.  Distinct laterality alterations distinguish mild cognitive impairment and Alzheimer's disease from healthy aging: Statistical parametric mapping with high resolution MRI , 2013, Human brain mapping.

[15]  Paul J. Laurienti,et al.  A permutation testing framework to compare groups of brain networks , 2013, Front. Comput. Neurosci..

[16]  John S. Allen,et al.  Normal neuroanatomical variation due to age: The major lobes and a parcellation of the temporal region , 2005, Neurobiology of Aging.

[17]  R. Coe,et al.  It's the Effect Size, Stupid What effect size is and why it is important , 2012 .

[18]  C. Fennema-Notestine,et al.  Effects of age on tissues and regions of the cerebrum and cerebellum , 2001, Neurobiology of Aging.

[19]  Thomas E. Nichols,et al.  Diagnosis and exploration of massively univariate neuroimaging models , 2003, NeuroImage.

[20]  M. Dwass.  Modified Randomization Tests for Nonparametric Hypotheses , 1957 .

[21]  Alan C Evans,et al.  Impact of scale space search on age‐ and gender‐related changes in MRI‐based cortical morphometry , 2013, Human brain mapping.

[22]  John Suckling,et al.  Power calculations for multicenter imaging studies controlled by the false discovery rate , 2010, Human brain mapping.

[23]  Stephen M. Smith,et al.  Permutation inference for the general linear model , 2014, NeuroImage.

[24]  Emiliano Macaluso,et al.  Grey and white matter changes at different stages of Alzheimer's disease. , 2010, Journal of Alzheimer's disease : JAD.

[25]  Roberto Viviani,et al.  Non-normality and transformations of random fields, with an application to voxel-based morphometry , 2007, NeuroImage.

[26]  R. Gur,et al.  Gender differences in age effect on brain atrophy measured by magnetic resonance imaging. , 1991, Proceedings of the National Academy of Sciences of the United States of America.

[27]  A. Dale,et al.  Effects of age on volumes of cortex, white matter and subcortical structures , 2005, Neurobiology of Aging.

[28]  Stephen M. Smith,et al.  Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm , 2001, IEEE Transactions on Medical Imaging.

[29]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[30]  Stephen M. Smith,et al.  Age-related changes in grey and white matter structure throughout adulthood , 2010, NeuroImage.

[31]  Joanna M. Wardlaw,et al.  Variance in Brain Volume with Advancing Age: Implications for Defining the Limits of Normality , 2013, PloS one.

[32]  Daniel S. Marcus,et al.  The extensible neuroimaging archive toolkit , 2007, Neuroinformatics.

[33]  F. Esposito,et al.  Microstructural Changes across Different Clinical Milestones of Disease in Amyotrophic Lateral Sclerosis , 2015, PloS one.

[34]  Thomas E. Nichols,et al.  Nonparametric permutation tests for functional neuroimaging: A primer with examples , 2002, Human brain mapping.

[35]  Karl J. Friston,et al.  Distributional Assumptions in Voxel-Based Morphometry , 2002, NeuroImage.

[36]  Mark W. Woolrich,et al.  Advances in functional and structural MR image analysis and implementation as FSL , 2004, NeuroImage.

[37]  Thomas E. Nichols,et al.  Thresholding of Statistical Maps in Functional Neuroimaging Using the False Discovery Rate , 2002, NeuroImage.

[38]  A. Represa,et al.  Causes and Consequences of Gray Matter Heterotopia , 2015, CNS neuroscience & therapeutics.