Quantile regression neural networks: Implementation in R and application to precipitation downscaling

The qrnn package for R implements the quantile regression neural network, which is an artificial neural network extension of linear quantile regression. The model formulation follows from previous work on the estimation of censored regression quantiles. The result is a nonparametric, nonlinear model suitable for making probabilistic predictions of mixed discrete-continuous variables like precipitation amounts, wind speeds, or pollutant concentrations, as well as continuous variables. A differentiable approximation to the quantile regression error function is adopted so that gradient-based optimization algorithms can be used to estimate model parameters. Weight penalty and bootstrap aggregation methods are used to avoid overfitting. For convenience, functions for quantile-based probability density, cumulative distribution, and inverse cumulative distribution functions are also provided. Package functions are demonstrated on a simple precipitation downscaling task.

[1]  Bernard Bobée,et al.  Frequency analysis of a sequence of dependent and/or non-stationary hydro-meteorological observations: a review , 2006 .

[2]  G. Cawley,et al.  Published online in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/joc.1318 DOWNSCALING HEAVY PRECIPITATION OVER THE UNITED KINGDOM: A COMPARISON OF DYNAMICAL AND STATISTICAL METHODS AND THEIR FUTURE SCENARIOS , 2006 .

[3]  M. Stephens EDF Statistics for Goodness of Fit and Some Comparisons , 1974 .

[4]  F. Vega-Redondo Complex Social Networks: Econometric Society Monographs , 2007 .

[5]  D. Baur,et al.  Modelling the effects of meteorological variables on ozone concentration—a quantile regression approach , 2004 .

[6]  Peter J. Huber,et al.  Wiley Series in Probability and Mathematics Statistics , 2005 .

[7]  R. Koenker,et al.  Goodness of Fit and Related Inference Processes for Quantile Regression , 1999 .

[8]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[9]  Gyógyszerészet Malnutrition in India , 2010 .

[10]  Alexander J. Smola,et al.  Nonparametric Quantile Estimation , 2006, J. Mach. Learn. Res..

[11]  V. Chernozhukov,et al.  QUANTILE AND PROBABILITY CURVES WITHOUT CROSSING , 2007, 0704.3649.

[12]  James W. Taylor A Quantile Regression Neural Network Approach to Estimating the Conditional Density of Multiperiod Returns , 2000 .

[13]  Stephen Jose Hanson,et al.  Minkowski-r Back-Propagation: Learning in Connectionist Models with Non-Euclidian Error Signals , 1987, NIPS.

[14]  Colin Chen A Finite Smoothing Algorithm for Quantile Regression , 2007 .

[15]  W. Loh,et al.  Nonparametric estimation of conditional quantiles using quantile regression trees ∗ ( Published in Bernoulli ( 2002 ) , 8 , 561 – 576 ) , 2008 .

[16]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[17]  W. Symes,et al.  Robust inversion of seismic data using the Huber norm , 2003 .

[18]  R. Koenker Censored Quantile Regression Redux , 2008 .

[19]  R. Reynolds,et al.  The NCEP/NCAR 40-Year Reanalysis Project , 1996, Renewable Energy.

[20]  R. Koenker,et al.  Quantile spline models for global temperature change , 1994 .

[21]  Alex J. Cannon,et al.  Downscaling recent streamflow conditions in British Columbia, Canada using ensemble neural network models , 2002 .

[22]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[23]  R. Koenker,et al.  Regression Quantiles , 2007 .

[24]  Alex J. Cannon,et al.  A graphical sensitivity analysis for statistical climate models: application to Indian monsoon rainfall prediction by artificial neural networks and multiple linear regression models , 2002 .

[25]  Alex J. Cannon,et al.  MODELING TRANSIENT pH DEPRESSIONS IN COASTAL STREAMS OF BRITISH COLUMBIA USING NEURAL NETWORKS 1 , 2001 .

[26]  Alex J. Cannon,et al.  Probabilistic Multisite Precipitation Downscaling by an Expanded Bernoulli–Gamma Density Network , 2008 .

[27]  John Bjørnar Bremnes,et al.  Probabilistic Forecasts of Precipitation in Terms of Quantiles Using NWP Model Output , 2004 .

[28]  P. Friederichs,et al.  Statistical Downscaling of Extreme Precipitation Events Using Censored Quantile Regression , 2007 .

[29]  M. W Gardner,et al.  Artificial neural networks (the multilayer perceptron)—a review of applications in the atmospheric sciences , 1998 .

[30]  Carl E. Rasmussen,et al.  Evaluating Predictive Uncertainty Challenge , 2005, MLCW.

[31]  Dan Cornford,et al.  Neural Network-Based Wind Vector Retrieval from Satellite Scatterometer Data , 1999, Neural Computing & Applications.

[32]  Achim Zeileis,et al.  Econometrics in R: Past, Present, and Future , 2008 .

[33]  Roger Koenker,et al.  Additive Models for Quantile Regression: An Analysis of Risk Factors for Malnutrition in India , 2010 .

[34]  John Bjørnar Bremnes,et al.  Probabilistic wind power forecasts using local quantile regression , 2004 .

[35]  Gareth J. Janacek,et al.  Predictive Uncertainty in Environmental Modelling , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[36]  Padraig Cunningham,et al.  Confidence and prediction intervals for neural network ensembles , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[37]  Gavin C. Cawley,et al.  Maximum likelihood cost functions for neural network models of air quality data , 2003 .

[38]  A. Raftery,et al.  Probabilistic forecasts, calibration and sharpness , 2007 .

[39]  Peter J. Huber,et al.  Robust Statistics , 2005, Wiley Series in Probability and Statistics.

[40]  B. Cade,et al.  A gentle introduction to quantile regression for ecologists , 2003 .

[41]  Bobby Schnabel,et al.  A modular system of algorithms for unconstrained minimization , 1985, TOMS.

[42]  R. Koenker,et al.  An interior point algorithm for nonlinear quantile regression , 1996 .

[43]  Tom Heskes,et al.  Practical Confidence and Prediction Intervals , 1996, NIPS.

[44]  William W. Hsieh,et al.  Applying Neural Network Models to Prediction and Data Analysis in Meteorology and Oceanography. , 1998 .

[45]  Marco Saerens,et al.  Building cost functions minimizing to some summary statistics , 2000, IEEE Trans. Neural Networks Learn. Syst..

[46]  Alex J. Cannon A flexible nonlinear modelling framework for nonstationary generalized extreme value analysis in hydroclimatology , 2010 .