What's normal anyway? Residual plots are more telling than significance tests when checking ANOVA assumptions

We consider two questions important for applying analysis of variance (ANOVA): Should normality be checked on the raw data or on the residuals (or is it immaterial which of the two approaches we take)? Should normality and homogeneity of variance be checked using significance tests or diagnostic plots (or both)? Based on two examples, we show that residuals should be used for model checking and that residual plots are better for checking ANOVA assumptions than statistical tests. We also discuss why one should be very cautious when using statistical tests to check the assumptions.

[1]  Karl Moder,et al.  The two-sample t test: pre-testing its assumptions does not pay off , 2011 .

[2]  H. Piepho.  Data Transformation in Statistical Analysis of Field Trials with Changing Treatment Variance , 2009 .

[3]  Michael D. Casler,et al.  Fundamentals of Experimental Design: Guidelines for Designing Successful Experiments , 2015 .

[4]  Hans-Peter Piepho,et al.  Checking Normality and Homoscedasticity in the General Linear Model Using Diagnostic Plots , 2012, Commun. Stat. Simul. Comput..

[5]  M. McIntosh,et al.  Can Analysis of Variance Be More Significant , 2015 .

[6]  W. R. Schucany,et al.  Preliminary Goodness-of-Fit Tests for Normality do not Validate the One-Sample Student t , 2006 .

[7]  Claus Thorn Ekstrøm.  Teaching "Instant Experience" with Graphical Model Validation Techniques. , 2014 .

[8]  Marcin Kozak,et al.  The effects of data input errors on subsequent statistical inference , 2015 .

[9]  Marla S. McIntosh,et al.  “ Is , or is not , the two great ends of Fate”: Errors in Agronomic Research , 2015 .

[10]  M. Kozak.  Analyzing one-way experiments: a piece of cake of a pain in the neck? , 2009 .

[11]  Hans-Peter Piepho,et al.  Why Randomize Agricultural Experiments , 2013 .

[12]  D G Altman,et al.  Statistics notes: Absence of evidence is not evidence of absence , 1995 .

[13]  B. Moser,et al.  Homogeneity of Variance in the Two-Sample Means Test , 1992 .

[14]  Achim Zeileis,et al.  Diagnostic Checking in Regression Relationships , 2015 .