Selection of smoothing parameter estimators for general regression neural networks - Applications to hydrological and water resources modelling

Multi-layer perceptron artificial neural networks are used extensively in hydrological and water resources modelling. However, a significant limitation with their application is that it is difficult to determine the optimal model structure. General regression neural networks (GRNNs) overcome this limitation, as their model structure is fixed. However, there has been limited investigation into the best way to estimate the parameters of GRNNs within water resources applications. In order to address this shortcoming, the performance of nine different estimation methods for the GRNN smoothing parameter is assessed in terms of accuracy and computational efficiency for a number of synthetic and measured data sets with distinct properties. Of these methods, five are based on bandwidth estimators used in kernel density estimation, and four are based on single and multivariable calibration strategies. In total, 5674 GRNN models are developed and preliminary guidelines for the selection of GRNN parameter estimation methods are provided and tested.

[1]  Holger R. Maier,et al.  Bayesian model selection applied to artificial neural networks used for water resources modeling , 2008 .

[2]  Ashu Jain,et al.  A comparative analysis of training methods for artificial neural network rainfall-runoff models , 2006, Appl. Soft Comput..

[3]  D. W. Scott,et al.  Biased and Unbiased Cross-Validation in Density Estimation , 1987 .

[4]  Donald F. Specht,et al.  Probabilistic neural networks , 1990, Neural Networks.

[5]  Holger R. Maier,et al.  Forecasting chlorine residuals in a water distribution system using a general regression neural network , 2006, Math. Comput. Model..

[6]  Holger R. Maier,et al.  Selection of input variables for data driven models: An average shifted histogram partial mutual information estimator approach , 2009 .

[7]  Holger R. Maier,et al.  Real‐time deployment of artificial neural network forecasting models: Understanding the range of applicability , 2012 .

[8]  Martin F. Lambert,et al.  Bayesian training of artificial neural networks used for water resources modeling , 2005 .

[9]  James Stephen Marron,et al.  Comparison of data-driven bandwith selectors , 1988 .

[10]  Holger R. Maier,et al.  The effect of internal parameters and geometry on the performance of back-propagation neural networks: an empirical study , 1998 .

[11]  Ashu Jain,et al.  Development of effective and efficient rainfall‐runoff models using integration of deterministic, real‐coded genetic algorithms and artificial neural network techniques , 2004 .

[12]  Martin F. Lambert,et al.  Calibration and validation of neural networks to ensure physically plausible hydrological modeling , 2005 .

[13]  Robert J. Abrahart,et al.  HydroTest: A web-based toolbox of evaluation metrics for the standardised assessment of hydrological forecasts , 2007, Environ. Model. Softw..

[14]  Tiesong Hu,et al.  River flow time series prediction with a range-dependent neural network , 2001 .

[15]  Holger R. Maier,et al.  Protocol for developing ANN models and its application to the assessment of the quality of the ANN model development process in drinking water quality modelling , 2014, Environ. Model. Softw..

[16]  Holger R. Maier,et al.  A method for comparing data splitting approaches for developing hydrological ANN models , 2012 .

[17]  Ashu Jain,et al.  Comparative Analysis of Event-Based Rainfall-Runoff Modeling Techniques—Deterministic, Statistical, and Artificial Neural Networks , 2003 .

[18]  Zongwu Cai,et al.  Weighted Nadaraya–Watson regression estimation , 2001 .

[19]  Martin D. Buhmann,et al.  Radial Basis Functions: Theory and Implementations: Preface , 2003 .

[20]  James Stephen Marron,et al.  Estimation of integrated squared density derivatives , 1987 .

[21]  William H. Press,et al.  Numerical Recipes: FORTRAN , 1988 .

[22]  Simon J. Sheather,et al.  Using non stochastic terms to advantage in kernel-based estimation of integrated squared density derivatives , 1991 .

[23]  J. B. Nixon,et al.  Investigation into the relationship between chlorine decay and water distribution parameters using data driven methods , 2006, Math. Comput. Model..

[24]  Wenyan Wu,et al.  A benchmarking approach for comparing data splitting methods for modeling water resources parameters using artificial neural networks , 2013 .

[25]  Holger R. Maier,et al.  Understanding the behaviour and optimising the performance of back-propagation neural networks: an empirical study , 1998 .

[26]  K. P. Sudheer,et al.  Methods used for the development of neural networks for the prediction of water resource variables in river systems: Current status and future directions , 2010, Environ. Model. Softw..

[27]  Holger R. Maier,et al.  Optimal division of data for neural network models in water resources applications , 2002 .

[28]  Joseph H. A. Guillaume,et al.  Characterising performance of environmental models , 2013, Environ. Model. Softw..

[29]  Hikmet Kerem Cigizoglu,et al.  Generalized regression neural network in modelling river sediment yield , 2006, Adv. Eng. Softw..

[30]  Rohana J. Karunamuni,et al.  On boundary correction in kernel density estimation , 2005 .

[31]  Holger R. Maier,et al.  Input determination for neural network models in water resources applications. Part 2. Case study: forecasting salinity in a river , 2005 .

[32]  James Stephen Marron,et al.  On the use of pilot estimators in bandwidth selection , 1992 .

[33]  Holger R. Maier,et al.  Data splitting for artificial neural networks using SOM-based stratified sampling , 2010, Neural Networks.

[34]  M. Rudemo.  Empirical Choice of Histograms and Kernel Density Estimators , 1982 .

[35]  Donald F. Specht,et al.  A general regression neural network , 1991, IEEE Trans. Neural Networks.

[36]  Holger R. Maier,et al.  Non-linear variable selection for artificial neural networks using partial mutual information , 2008, Environ. Model. Softw..

[37]  Christian W. Dawson,et al.  Evaluation of artificial neural network techniques for flow forecasting in the River Yangtze , China 619 , 2002 .

[38]  Holger R. Maier,et al.  Application of Artificial Neural Networks to Forecasting of Surface Water Quality Variables: Issues, Applications and Challenges , 2000 .

[39]  P. Krause,et al.  COMPARISON OF DIFFERENT EFFICIENCY CRITERIA FOR HYDROLOGICAL MODEL ASSESSMENT , 2005 .

[40]  Holger R. Maier,et al.  Empirical comparison of various methods for training feed‐Forward neural networks for salinity forecasting , 1999 .

[41]  T. Cacoullos.  Estimation of a multivariate density , 1966 .

[42]  J. Marron,et al.  Smoothed cross-validation , 1992 .

[43]  P. Coulibaly,et al.  Two decades of anarchy? Emerging themes and outstanding challenges for neural network river forecasting , 2012 .

[44]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[45]  Holger R. Maier,et al.  Input determination for neural network models in water resources applications. Part 1—background and methodology , 2005 .

[46]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[47]  A. Castelletti,et al.  Tree‐based iterative input variable selection for hydrological modeling , 2013 .

[48]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[49]  H. Maier,et al.  The Use of Artificial Neural Networks for the Prediction of Water Quality Parameters , 1996 .

[50]  B. Bobée,et al.  Improving extreme hydrologic events forecasting using a new criterion for artificial neural network selection , 2001 .

[51]  A. Bowman.  An alternative method of cross-validation for the smoothing of density estimates , 1984 .

[52]  Faming Liang,et al.  Explicitly integrating parameter, input, and structure uncertainties into Bayesian Neural Networks for probabilistic hydrologic forecasting , 2011 .

[53]  David W. Scott,et al.  Multivariate Density Estimation and Visualization , 2012 .