Small-sample performance and underlying assumptions of a bootstrap-based inference method for a general analysis of covariance model with possibly heteroskedastic and nonnormal errors

It is well known that the standard F test is severely affected by heteroskedasticity in unbalanced analysis of covariance models. Currently available potential remedies for such a scenario are based on heteroskedasticity-consistent covariance matrix estimation (HCCME). However, the HCCME approach tends to be liberal in small samples. Therefore, in the present paper, we propose a combination of HCCME and a wild bootstrap technique, with the aim of improving the small-sample performance. We precisely state a set of assumptions for the general analysis of covariance model and discuss their practical interpretation in detail, since this issue may have been somewhat neglected in applied research so far. We prove that these assumptions are sufficient to ensure the asymptotic validity of the combined HCCME-wild bootstrap analysis of covariance. The results of our simulation study indicate that our proposed test remedies the problems of the analysis of covariance F test and its heteroskedasticity-consistent alternatives in small to moderate sample size scenarios. Our test only requires very mild conditions, thus being applicable in a broad range of real-life settings, as illustrated by the detailed discussion of a dataset from preclinical research on spinal cord injury. Our proposed method is ready-to-use and allows for valid hypothesis testing in frequently encountered settings (e.g., comparing group means while adjusting for baseline measurements in a randomized controlled clinical trial).

[1]  A. Bathke.  A Nonparametric Alternative to Analysis of Covariance , 2003 .

[2]  D. Dey,et al.  A First Course in Linear Model Theory , 2001 .

[3]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[4]  M. Craggs,et al.  Aberrant reflexes and function of the pelvic organs following spinal cord injury in man , 2006, Autonomic Neuroscience.

[5]  D K Kimura,et al.  Testing nonlinear regression parameters under heteroscedastic, normally distributed errors. , 1990, Biometrics.

[6]  Chunpeng Fan,et al.  Rank repeated measures analysis of covariance , 2017 .

[7]  G. Glass,et al.  Consequences of Failure to Meet Assumptions Underlying the Fixed Effects Analyses of Variance and Covariance , 1972 .

[8]  J. Ashford,et al.  Generalised covariance analysis with unequal error variances. , 1969, Biometrics.

[9]  Tong Zhu,et al.  An optimized wild bootstrap method for evaluation of measurement uncertainties of DTI-derived parameters in human brain , 2008, NeuroImage.

[10]  I Grant,et al.  Analysis of covariance as a remedy for demographic mismatch of research subject groups: some sobering simulations. , 1985, Journal of clinical and experimental neuropsychology.

[11]  Carl J. Huberty,et al.  Statistical Practices of Educational Researchers: An Analysis of their ANOVA, MANOVA, and ANCOVA Analyses , 1998 .

[12]  F. Cruz,et al.  Spinal Cord Injury and Bladder Dysfunction: New Ideas about an Old Problem , 2011, TheScientificWorldJournal.

[13]  S. Pocock,et al.  Subgroup analysis, covariate adjustment and baseline comparisons in clinical trial reporting: current practiceand problems , 2002, Statistics in medicine.

[14]  M. Murray,et al.  Lower urinary tract function in spinal cord-injured rats: midthoracic contusion versus transection , 2014, Spinal Cord.

[15]  S. Greenhouse,et al.  Adjusting for demographic covariates by the analysis of covariance. , 1992, Journal of clinical and experimental neuropsychology.

[16]  Changbao Wu,et al.  Jackknife, Bootstrap and Other Resampling Methods in Regression Analysis , 1986 .

[17]  Emmanuel Flachaire,et al.  The wild bootstrap, tamed at last , 2001 .

[18]  A. Hayes,et al.  Using heteroskedasticity-consistent standard error estimators in OLS regression: An introduction and software implementation , 2007, Behavior research methods.

[19]  Ernesto Zacur,et al.  Statistical analysis of relative pose information of subcortical nuclei: Application on ADNI data , 2011, NeuroImage.

[20]  James G. MacKinnon,et al.  Thirty Years of Heteroskedasticity-Robust Inference , 2013 .

[21]  G G Koch,et al.  Non‐parametric analysis of covariance for confirmatory randomized clinical trials to evaluate dose–response relationships , 2001, Statistics in medicine.

[22]  Regina Y. Liu.  Bootstrap Procedures under some Non-I.I.D. Models , 1988 .

[23]  G G Koch,et al.  Issues for covariance analysis of dichotomous and ordered categorical data from randomized clinical trials and non-parametric strategies for addressing them. , 1998, Statistics in medicine.

[24]  S. Owen,et al.  Uses and abuses of the analysis of covariance. , 1998, Research in nursing & health.

[25]  Francisco Cribari Neto.  Asymptotic inference under heteroskedasticity of unknown form , 2004 .

[26]  H. White.  A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity , 1980 .

[27]  S. M. Sadooghi-Alvandi,et al.  A Parametric Bootstrap Approach for One-Way ANCOVA with Unequal Variances , 2013 .

[28]  Kristin E. Porter,et al.  Robustness of ordinary least squares in randomized clinical trials , 2016, Statistics in medicine.

[29]  J. B. Nielsen.  Lower urinary tract function in vesicoureteral reflux. , 1989, Scandinavian journal of urology and nephrology. Supplementum.

[30]  R. K. Misra.  A Multivariate Procedure for Comparing Mean Vectors for Populations with Unequal Regression Coefficient and Residual Covariance Matrices , 1996 .

[31]  E. Mammen.  Bootstrap and Wild Bootstrap for High Dimensional Linear Models , 1993 .

[32]  L. A. van der Ark,et al.  Comment on Thas, De Neve, Clement and Ottoy (2012): Probabilistic index models , 2012 .

[33]  An efficient analysis of covariance model for crossover thorough QT studies with period‐specific pre‐dose baselines , 2014, Pharmaceutical statistics.

[34]  Markus Pauly,et al.  Weak Convergence of the Wild Bootstrap for the Aalen–Johansen Estimator of the Cumulative Incidence Function of a Competing Risk , 2013 .

[35]  M. Davidian,et al.  Covariate adjustment for two‐sample treatment comparisons in randomized clinical trials: A principled yet flexible approach , 2008, Statistics in medicine.

[36]  Bradley E. Huitema,et al.  The analysis of covariance and alternatives , 1980 .

[37]  Stephen Senn,et al.  Change from baseline and analysis of covariance revisited , 2006, Statistics in medicine.

[38]  Keith M Godfrey,et al.  Correction of unexpected distributions of P values from analysis of whole genome arrays by rectifying violation of statistical assumptions , 2012, BMC Genomics.

[39]  H. White,et al.  Some heteroskedasticity-consistent covariance matrix estimators with improved finite sample properties☆ , 1985 .

[40]  E. Lesaffre,et al.  A note on non‐parametric ANCOVA for covariate adjustment in randomized clinical trials , 2003, Statistics in medicine.

[41]  P. Chaussé,et al.  A Simulation-Based Comparison of Covariate Adjustment Methods for the Analysis of Randomized Controlled Trials , 2016, International journal of environmental research and public health.

[42]  L. Clement,et al.  Probabilistic index models , 2012 .