Robust tests for multivariate factorial designs under heteroscedasticity

The question of how to analyze several multivariate normal mean vectors when normality and covariance homogeneity assumptions are violated is considered in this article. For the two-way MANOVA layout, we address this problem adapting results presented by Brunner, Dette, and Munk (BDM; 1997) and Vallejo and Ato (modified Brown–Forsythe [MBF]; 2006) in the context of univariate factorial and split-plot designs and a multivariate version of the linear model (MLM) to accommodate heterogeneous data. Furthermore, we compare these procedures with the Welch–James (WJ) approximate degrees of freedom multivariate statistics based on ordinary least squares via Monte Carlo simulation. Our numerical studies show that of the methods evaluated, only the modified versions of the BDM and MBF procedures were robust to violations of underlying assumptions. The MLM approach was only occasionally liberal, and then by only a small amount, whereas the WJ procedure was often liberal if the interactive effects were involved in the design, particularly when the number of dependent variables increased and total sample size was small. On the other hand, it was also found that the MLM procedure was uniformly more powerful than its most direct competitors. The overall success rate was 22.4% for the BDM, 36.3% for the MBF, and 45.0% for the MLM.

[1]  Jiin-Huarng Guo,et al.  Improved Robust Test Statistic Based on Trimmed Means and Hall's Transformation for Two-way ANOVA Models Under Non-normality , 2004 .

[2]  Chester L. Olson,et al.  Comparative Robustness of Six Tests in Multivariate Analysis of Variance , 1974 .

[3]  J. Hair.  Multivariate data analysis : a global perspective , 2010 .

[4]  G. Box.  Some Theorems on Quadratic Forms Applied in the Study of Analysis of Variance Problems, I. Effect of Inequality of Variance in the One-Way Classification , 1954 .

[5]  R. Wilcox.  Introduction to Robust Estimation and Hypothesis Testing , 1997 .

[6]  Manuel Ato García,et al.  Comparative Robustness of Recent Methods for Analyzing Multivariate Repeated Measures Designs , 2007 .

[7]  A. W. Blöte,et al.  Students' flexibility in solving two-digit addition and subtraction problems : Instruction effects , 2001 .

[8]  E. Ziegel.  Introduction to Robust Estimation and Hypothesis Testing (2nd ed.) , 2005 .

[9]  G. S. James.  THE COMPARISON OF SEVERAL GROUPS OF OBSERVATIONS WHEN THE RATIOS OF THE POPULATION VARIANCES ARE UNKNOWN , 1951 .

[10]  Pablo Livacic-Rojas,et al.  Analysis of unbalanced factorial designs with heteroscedastic data , 2010 .

[11]  Carl J. Huberty,et al.  Statistical Practices of Educational Researchers: An Analysis of their ANOVA, MANOVA, and ANCOVA Analyses , 1998 .

[12]  Holger Dette,et al.  Box-Type Approximations in Nonparametric Factorial Designs , 1997 .

[13]  T. Micceri.  The unicorn, the normal curve, and other improbable creatures. , 1989 .

[14]  Janet K. Sheehan-Holt.  MANOVA Simultaneous Test Procedures: The Power and Robustness of Restricted Multivariate Contrasts , 1998 .

[15]  Some Alternative Approximate Tests for a Split Plot Design. , 1994, Multivariate behavioral research.

[16]  K. Sassenberg,et al.  Knowing how to react: automatic response priming from social categories. , 2006, Journal of personality and social psychology.

[17]  M. Kenward,et al.  Parametric modelling of growth curve data: An overview , 2001 .

[18]  John C. Lind,et al.  Two-sample T–2 procedure and the assumption of homogeneous covariance matrices. , 1979 .

[19]  M. Kenward,et al.  Small sample inference for fixed effects from restricted maximum likelihood. , 1997, Biometrics.

[20]  Michael G. Kenward,et al.  An improved approximation to the precision of fixed effects from restricted maximum likelihood , 2009, Comput. Stat. Data Anal..

[21]  Thomas Mathew,et al.  Generalized p -values and generalized confidence regions for the multivariate Behrens-Fisher problem and MANOVA , 2004 .

[22]  K. Krishnamoorthy,et al.  A parametric bootstrap solution to the MANOVA under heteroscedasticity , 2010 .

[23]  T. Davenport,et al.  Performance of Monte Carlo Permutation and Approximate Tests for Multivariate Means Comparisons With Small Sample Sizes When Parametric Assumptions are Violated , 2009 .

[24]  Ron D Hays,et al.  Correlated physical and mental health summary scores for the SF-36 and SF-12 Health Survey, V.1 , 2007, Health and quality of life outcomes.

[25]  H. Keselman,et al.  Robust and powerful nonorthogonal analyses , 1995 .

[26]  James Algina,et al.  A generally robust approach for testing hypotheses and setting confidence intervals for effect sizes. , 2008, Psychological methods.

[27]  Kanti V. Mardia,et al.  The effect of nonnormality on some multivariate tests and robustness to nonnormality in the linear model , 1971 .

[28]  M. Puri,et al.  The multivariate nonparametric Behrens–Fisher problem , 2002 .

[29]  Cora J. M. Maas,et al.  The Multilevel Approach to Repeated Measures for Complete and Incomplete Data , 2003 .

[30]  Solomon W. Harrar,et al.  How to compare small multivariate samples using nonparametric tests , 2008, Comput. Stat. Data Anal..

[31]  H. Bekkering,et al.  Object manipulation and motion perception: evidence of an influence of action planning on visual processing. , 2009, Journal of experimental psychology. Human perception and performance.

[32]  B. L. Welch.  ON THE COMPARISON OF SEVERAL MEAN VALUES: AN ALTERNATIVE APPROACH , 1951 .

[33]  Ellián Tuero-Herrero,et al.  Selecting the best unbalanced repeated measures model , 2011, Behavior research methods.

[34]  Mortaza Jamshidian,et al.  Testing equality of covariance matrices when data are incomplete , 2007, Comput. Stat. Data Anal..

[35]  A. J. Collins,et al.  Introduction To Multivariate Analysis , 1981 .

[36]  S. Johansen,et al.  The Welch - James approximation to the distribution of the residual sum of squares in a weighted lin , 1980 .

[37]  G. Vallejo,et al.  A robust approach for analyzing unbalanced factorial designs with fixed levels , 2010, Behavior research methods.

[38]  P. K. Ito.  7 Robustness of ANOVA and MANOVA test procedures , 1980 .

[39]  H. Keselman,et al.  Analyzing Multivariate Repeated Measures Designs: A Comparison of Two Approximate Degrees of Freedom Procedures , 2003, Multivariate behavioral research.

[40]  Scott E. Maxwell,et al.  Designing Experiments and Analyzing Data: A Model Comparison Perspective , 1990 .

[41]  Guillermo Vallejo,et al.  Modified Brown–Forsythe Procedure for Testing Interaction Effects in Split-Plot Designs , 2006, Multivariate behavioral research.

[42]  A. M. Mathai,et al.  Quadratic forms in random variables : theory and applications , 1992 .

[43]  R. Arvey,et al.  Small sample profile analysis with many variables. , 1982 .

[44]  B. Ambridge,et al.  The structure of working memory from 4 to 15 years of age. , 2004, Developmental psychology.

[45]  J. Algina,et al.  New Test Statistics for MANOVA/Descriptive Discriminant Analysis , 1996 .

[46]  Richard A. Johnson,et al.  Applied Multivariate Statistical Analysis , 1983 .

[47]  T. Beauchaine,et al.  Parent-child interactions, peripheral serotonin, and self-inflicted injury in adolescents. , 2008, Journal of consulting and clinical psychology.

[48]  D. Mehrotra.  Improving the brown-forsythe solution to the generalized behrens-fisher problem , 1997 .

[49]  Xiaonan Xue,et al.  The relationship between specific cognitive functions and falls in aging. , 2007, Neuropsychology.

[50]  D. G. Nel.  Tests for Equality of Parameter Matrices in Two Multivariate Linear Models , 1997 .