On robust testing for normality in chemometrics

Abstract The assumption that the data has been generated by a normal distribution underlies many statistical methods used in chemometrics. While such methods can be quite robust to small deviations from normality, for instance caused by a small number of outliers, common tests for normality are not and will often needlessly reject normality. It is therefore better to use tests from the little-known class of robust tests for normality. We illustrate the need for robust normality testing in chemometrics with several examples, review a class of robustified omnibus Jarque–Bera tests and propose a new class of robustified directed Lin–Mudholkar tests. The robustness and power of several tests for normality are compared in a large simulation study. The new tests are robust and have high power in comparison with both classic tests and other robust tests. A new graphical method for assessing normality is also introduced.

[1]  E. L. Lehmann,et al.  Descriptive Statistics for Nonparametric Models II. Location , 1975 .

[2]  Weiwen Miao,et al.  The Importance of Checking the Assumptions Underlying Statistical Analysis: Graphical Methods for Assessing Normality , 2005 .

[3]  Narayanaswamy Balakrishnan,et al.  Permanents, Order Statistics, Outliers, and Robustness , 2007 .

[4]  V. A. Uthoff,et al.  The Most Powerful Scale and Location Invariant Test of the Normal Versus the Double Exponential , 1973 .

[5]  T. Hettmansperger,et al.  Robust Nonparametric Statistical Methods , 1998 .

[6]  J. Ord,et al.  Characterization Problems in Mathematical Statistics , 1975 .

[7]  Anil K. Bera,et al.  Efficient tests for normality, homoscedasticity and serial independence of regression residuals: Monte Carlo Evidence , 1981 .

[8]  E. S. Pearson,et al.  Tests for departure from normality. Empirical results for the distributions of b2 and √b1 , 1973 .

[9]  Ghanshyam L. Vaghjiani,et al.  New measurement of the rate coefficient for the reaction of OH with methane , 1991, Nature.

[10]  J. Marden,et al.  Positions and QQ Plots , 2004 .

[11]  J. Miller,et al.  Statistics and chemometrics for analytical chemistry , 2005 .

[12]  Carlos M. Urzúa.  On the correct use of omnibus tests for normality , 1996 .

[13]  Richard G. Brereton,et al.  Applied Chemometrics for Scientists , 2007 .

[14]  G. S. Mudholkar,et al.  Independence characterizations and testing normality against restricted skewness–kurtosis alternatives , 2002 .

[15]  Steven G. Gilmour,et al.  Optimum design of experiments for statistical inference , 2012 .

[16]  M. Stehlík,et al.  Modeling methane emission via the infinite moving average process , 2013 .

[17]  Anja Struyf,et al.  Goodness-of-fit tests based on a robust measure of skewness , 2008, Comput. Stat..

[18]  Juan Antonio Cuesta-Albertos,et al.  Assessing when a sample is mostly normal , 2010, Comput. Stat. Data Anal..

[19]  P. Hall.  The Bootstrap and Edgeworth Expansion , 1992 .

[20]  J. Gastwirth,et al.  A robust modification of the Jarque–Bera test of normality , 2008 .

[21]  Ching-Chuong Lin,et al.  A simple test for normality against asymmetric alternatives , 1980 .

[22]  Måns Eriksson.  Two simple tests for normality with high power , 2010 .

[23]  Kalliopi Mylona,et al.  Optimum design of experiments for statistical inference : discussion , 2012 .

[24]  Milan Stehlík,et al.  Filling and D-optimal Designs for the Correlated Generalized Exponential Model , 2012 .

[25]  H. Lilliefors.  On the Kolmogorov-Smirnov Test for Normality with Mean and Variance Unknown , 1967 .

[26]  Milan Stehlík,et al.  Small Sample Robust Testing for Normality against Pareto Tails , 2012, Commun. Stat. Simul. Comput..

[27]  R. Geary.  THE RATIO OF THE MEAN DEVIATION TO THE STANDARD DEVIATION AS A TEST OF NORMALITY , 1935 .

[28]  D. Darling,et al.  A Test of Goodness of Fit , 1954 .

[29]  Måns Thulin,et al.  Tests for multivariate normality based on canonical correlations , 2011, Stat. Methods Appl..

[30]  H. Büning,et al.  Jarque–Bera Test and its Competitors for Testing Normality – A Power Comparison , 2007 .

[31]  G. Box,et al.  Response Surfaces, Mixtures and Ridge Analyses , 2007 .

[32]  S. Shapiro,et al.  An Analysis of Variance Test for Normality (Complete Samples) , 1965 .

[33]  H. Thode.  Testing For Normality , 2002 .

[34]  Yulia R. Gel,et al.  lawstat: An R Package for Law, Public Policy and Biostatistics , 2008 .

[35]  Weiwen Miao,et al.  Robust directed tests of normality against heavy-tailed alternatives , 2007, Comput. Stat. Data Anal..

[36]  Tingting Li,et al.  CH4MODwetland: A biogeophysical model for simulating methane emissions from natural wetlands , 2010 .

[37]  S. Stigler.  The Asymptotic Distribution of the Trimmed Mean , 1973 .

[38]  M. Thulin.  On two simple tests for normality with high power , 2010, 1008.5319.

[39]  P. Bickel,et al.  DESCRIPTIVE STATISTICS FOR NONPARAMETRIC MODELS IV. SPREAD , 1979 .

[40]  Mia Hubert,et al.  Robustness and Outlier Detection in Chemometrics , 2006 .

[41]  N. L. Carr.  Kinetics of Catalytic Isomerization of n-Pentane , 1960 .