Comparing Spatial Predictions

Under a general loss function, we develop a hypothesis test to determine whether a significant difference in the spatial predictions produced by two competing models exists on average across the entire spatial domain of interest. The null hypothesis is that of no difference, and a spatial loss differential is created based on the observed data, the two sets of predictions, and the loss function chosen by the researcher. The test assumes only isotropy and short-range spatial dependence of the loss differential but does allow it to be non-Gaussian, non-zero-mean, and spatially correlated. Constant and nonconstant spatial trends in the loss differential are treated in two separate cases. Monte Carlo simulations illustrate the size and power properties of this test, and an example based on daily average wind speeds in Oklahoma is used for illustration. Supplemental results are available online.

[1]  Thomas A. Herring,et al.  Blowing in the wind , 1988, Nature.

[2]  Weile Wang,et al.  Intraseasonal Interactions between Temperature and Vegetation over the Boreal Forests , 2007 .

[3]  Douglas W. Nychka,et al.  Nonstationary covariance modeling for incomplete data: Monte Carlo EM approach , 2011, Comput. Stat. Data Anal..

[4]  B. Whitcher,et al.  Multiple hypothesis mapping of functional MRI data in orthogonal and complex wavelet domains , 2005, IEEE Transactions on Signal Processing.

[5]  Carol A. Gotway,et al.  Statistical Methods for Spatial Data Analysis , 2004 .

[6]  Parameter estimation and tests of equal forecast accuracy between non-nested models , 2004 .

[7]  Xiaotong Shen,et al.  Nonparametric Hypothesis Testing for a Spatial Signal , 2002 .

[8]  Tae Yoon Kim,et al.  Practically Applicable Central Limit Theorem for Spatial Statistics , 2009 .

[9]  Peter Nijkamp,et al.  Forecasting Regional Labor Market Developments under Spatial Autocorrelation , 2007 .

[10]  Paul Newbold,et al.  Testing the equality of prediction mean squared errors , 1997 .

[11]  Loïk Berre,et al.  A Wavelet Approach to Representing Background Error Covariances in a Limited-Area Model , 2005 .

[12]  Richard A. Davis,et al.  Time Series: Theory and Methods , 2013 .

[13]  T. Gneiting.  Quantiles as Optimal Point Predictors , 2008 .

[14]  Yuhong Yang,et al.  Nonparametric Regression with Correlated Errors , 2001 .

[15]  M. Genton,et al.  Powering Up With Space-Time Wind Forecasting , 2010 .

[16]  R. Carroll,et al.  Variogram estimation in the presence of trend. , 2012, Statistics and its interface.

[17]  Donald B. Percival,et al.  Three Curious Properties of the Sample Variance and Autocovariance for Stationary Processes With Unknown Mean , 1993 .

[18]  Y. Benjamini,et al.  False Discovery Rates for Spatial Signals , 2007 .

[19]  Sucharita Gopal,et al.  Spatial Interpolation of Surface Air Temperatures Using Artificial Neural Networks: Evaluating Their Use for Downscaling GCMs , 2000 .

[20]  J. McBride,et al.  Verification of precipitation in weather systems: determination of systematic errors , 2000 .

[21]  Halbert White,et al.  Tests of Conditional Predictive Ability , 2003 .

[22]  Noel A Cressie,et al.  Global statistical analysis of MISR aerosol data: a massive data product from NASA's Terra satellite , 2007 .

[23]  J. Andrew Royle,et al.  Multiresolution models for nonstationary spatial covariance functions , 2002 .

[24]  H Lee Willis,et al.  Spatial electric load forecasting , 1996 .

[25]  T. Gneiting.  Quantiles as optimal point forecasts , 2011 .

[26]  F. Scholz.  Maximum Likelihood Estimation , 2006 .

[27]  Noel A Cressie,et al.  When are relative variograms useful in geostatistics? , 1985 .

[28]  K. West,et al.  Asymptotic Inference about Predictive Ability , 1996 .

[29]  N. Cressie.  Fitting variogram models by weighted least squares , 1985 .

[30]  Adrian E. Raftery,et al.  Locally Calibrated Probabilistic Temperature Forecasting Using Geostatistical Model Averaging and Local Bayesian Model Averaging , 2011 .

[31]  F. Diebold,et al.  Comparing Predictive Accuracy , 1994, Business Cycles.

[32]  E. Ronchetti,et al.  Robust Tests of Predictive Accuracy , 2002 .

[33]  Jean D. Opsomer,et al.  Smoothing parameter selection methods for nonparametric regression with spatially correlated errors , 2005 .

[34]  K. Mardia,et al.  Maximum likelihood estimation of models for residual covariance in spatial regression , 1984 .

[35]  Eric Gilleland,et al.  Verifying Forecasts Spatially , 2010 .

[36]  Jeffrey D. Hart,et al.  Some automated methods of smoothing time-dependent data , 1996 .

[37]  C. F. Sirmans,et al.  Nonstationary multivariate process modeling through spatially varying coregionalization , 2004 .

[38]  F. Atger,et al.  Spatial and Interannual Variability of the Reliability of Ensemble-Based Probabilistic Forecasts: Consequences for Calibration , 2003 .

[39]  Noel A Cressie,et al.  Detecting signals in FMRI data using powerful FDR procedures , 2008 .

[40]  A. Barnston,et al.  The Effect of Spatial Aggregation on the Skill of Seasonal Precipitation Forecasts , 2003 .