Controlling the Proportion of Falsely Rejected Hypotheses when Conducting Multiple Tests with Climatological Data

Abstract The analysis of climatological data often involves statistical significance testing at many locations. While the field significance approach determines if a field as a whole is significant, a multiple testing procedure determines which particular tests are significant. Many such procedures are available, most of which control, for every test, the probability of detecting significance that does not really exist. The aim of this paper is to introduce the novel “false discovery rate” approach, which controls the false rejections in a more meaningful way. Specifically, it controls a priori the expected proportion of falsely rejected tests out of all rejected tests; additionally, the test results are more easily interpretable. The paper also investigates the best way to apply a false discovery rate (FDR) approach to spatially correlated data, which are common in climatology. The most straightforward method for controlling the FDR makes an assumption of independence between tests, while other FDR-contr...

[1]  L. Wasserman,et al.  A stochastic process approach to false discovery control , 2004, math/0406519.

[2]  Y. Benjamini,et al.  Resampling-based false discovery rate controlling multiple test procedures for correlated test statistics , 1999 .

[3]  H. Storch,et al.  Statistical Analysis in Climate Research , 2000 .

[4]  L. Wasserman,et al.  Operating characteristics and extensions of the false discovery rate procedure , 2002 .

[5]  Mike Rees,et al.  5. Statistics for Spatial Data , 1993 .

[6]  Daniel S. Wilks,et al.  Resampling Hypothesis Tests for Autocorrelated Fields , 1997 .

[7]  Richard W. Katz,et al.  Sir Gilbert Walker and a Connection between El Niño and Statistics , 2002 .

[8]  V. Ventura,et al.  Multiple Indices of Northern Hemisphere Cyclone Activity, Winters 1949–99 , 2002 .

[9]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[10]  Y. Benjamini,et al.  THE CONTROL OF THE FALSE DISCOVERY RATE IN MULTIPLE TESTING UNDER DEPENDENCY , 2001 .

[11]  Daniel S. Wilks,et al.  Statistical Methods in the Atmospheric Sciences: An Introduction , 1995 .

[12]  G. Casella,et al.  Statistical Inference , 2003, Encyclopedia of Social Network Analysis and Mining.

[13]  Noel A Cressie,et al.  Statistics for Spatial Data. , 1992 .

[14]  R. E. Livezey,et al.  Statistical Field Significance and its Determination by Monte Carlo Techniques , 1983 .

[15]  John D. Storey.  A direct approach to false discovery rates , 2002 .