Optimal design of artificial neural networks by a multi-objective strategy: groundwater level predictions

Abstract Currently, environmental modelling is frequently conducted with the aid of artificial neural networks (ANNs) in an effort to achieve greater accuracy in simulation and forecasting beyond that typically obtained when using solely linear models. For the design of an ANN, modellers must contend with two key issues: (a) the selection of model input and (b) the determination of the number of hidden neurons. A novel approach is introduced to address the optimal design of ANNs based on a multi-objective strategy that enables the user to find a set of feasible ANNs, determined as optimal trade-off solutions between model simplicity and accuracy. This is achieved in a multi-objective fashion by simultaneously minimizing three different cost functions: the model input dimension, the hidden neuron number and the generalization error computed on a validation set of data. The multi-objective approach is based on the Pareto dominance criterion and an evolutionary strategy has been employed to solve the combinatorial optimization problem. From a theoretical perspective, the choice of a multi-objective approach marks an attempt to account for, and overcome, the “curse of dimensionality” and to circumvent the drawbacks of “overfitting” that are inherent in ANNs. Moreover, it is demonstrated that the strategy renders the choice of the ANN more robust, as is evident by “unseen data” in the testing stage, since structure determination is not merely based on the statistical evaluation of the generalization performance. The methodology is tested and the results are reported in a case study relating groundwater level predictions to total monthly rainfall.

[1]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[2]  David Corne,et al.  The Pareto archived evolution strategy: a new baseline algorithm for Pareto multiobjective optimisation , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[3]  Stefano Alvisi,et al.  A short-term, pattern-based model for water-demand forecasting , 2006 .

[4]  S. Thomas Ng,et al.  A Modified Neural Network for Improving River Flow Prediction/Un Réseau de Neurones Modifié pour Améliorer la Prévision de L'Écoulement Fluvial , 2005 .

[5]  A. Soldati,et al.  Artificial neural network approach to flood forecasting in the River Arno , 2003 .

[6]  Adam P. Piotrowski,et al.  Are Artificial Neural Network Techniques Relevant for the Estimation of Longitudinal Dispersion Coefficient in Rivers , 2005 .

[7]  Christian W. Dawson,et al.  An artificial neural network approach to rainfall-runoff modelling , 1998 .

[8]  C. Fonseca,et al.  GENETIC ALGORITHMS FOR MULTI-OBJECTIVE OPTIMIZATION: FORMULATION, DISCUSSION, AND GENERALIZATION , 1993 .

[9]  Özgür Kişi,et al.  Multi-layer perceptrons with Levenberg-Marquardt training algorithm for suspended sediment concentration prediction and estimation / Prévision et estimation de la concentration en matières en suspension avec des perceptrons multi-couches et l’algorithme d’apprentissage de Levenberg-Marquardt , 2004 .

[10]  Joos Vandewalle,et al.  Modelling and forecasting of hydrological variables using artificial neural networks: the Kafue River sub-basin , 2003 .

[11]  Roland K. Price,et al.  A neural network model of rainfall-runoff using radial basis functions , 1996 .

[12]  Kuolin Hsu,et al.  Artificial Neural Network Modeling of the Rainfall‐Runoff Process , 1995 .

[13]  David Hinkley,et al.  Bootstrap Methods: Another Look at the Jackknife , 2008 .

[14]  Kalyanmoy Deb,et al.  Muiltiobjective Optimization Using Nondominated Sorting in Genetic Algorithms , 1994, Evolutionary Computation.

[15]  U. C. Kothyari,et al.  Artificial neural networks for daily rainfall—runoff modelling , 2002 .

[16]  O. Ks.  Multi-layer perceptrons with Levenberg-Marquardt training algorithm for suspended sediment concentration prediction and estimation , 2004 .

[17]  Charles Gide,et al.  Cours d'économie politique , 1911 .

[18]  Marco Laumanns,et al.  SPEA2: Improving the strength pareto evolutionary algorithm , 2001 .

[19]  R Govindaraju,et al.  ARTIFICIAL NEURAL NETWORKS IN HYDROLOGY: II, HYDROLOGIC APPLICATIONS , 2000 .

[20]  Robert J. Abrahart,et al.  Using pruning algorithms and genetic algorithms to optimise network architectures and forecasting inputs in a neural network rainfall-runoff model , 1999 .

[21]  Tiesong Hu,et al.  A Modified Neural Network for Improving River Flow Prediction , 2005 .

[22]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[23]  Gary B. Lamont,et al.  Multiobjective Evolutionary Algorithms: Analyzing the State-of-the-Art , 2000, Evolutionary Computation.

[24]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[25]  Anthony W. Minns,et al.  Subsymbolic methods for data mining in hydraulic engineering , 2000 .

[26]  Lothar Thiele,et al.  Multiobjective Optimization Using Evolutionary Algorithms - A Comparative Case Study , 1998, PPSN.

[27]  Peter J. Fleming,et al.  Multiobjective optimization and multiple constraint handling with evolutionary algorithms. I. A unified formulation , 1998, IEEE Trans. Syst. Man Cybern. Part A.

[28]  Dragan Savic,et al.  Single-objective vs. Multiobjective Optimisation for Integrated Decision Support , 2002 .

[29]  K. Lam,et al.  River flow time series prediction with a range-dependent neural network , 2001 .

[30]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[31]  R. Abrahart,et al.  Detection of conceptual model rainfall—runoff processes inside an artificial neural network , 2003 .

[32]  Jaroslaw J. Napiorkowski,et al.  Are artificial neural network techniques relevant for the estimation of longitudinal dispersion coefficient in rivers? / Les techniques de réseaux de neurones artificiels sont-elles pertinentes pour estimer le coefficient de dispersion longitudinale en rivières? , 2005 .

[33]  Orazio Giustolisi,et al.  Using a multi-objective genetic algorithm for SVM construction , 2006 .

[34]  Zbigniew Michalewicz,et al.  Evolutionary Computation 2 , 2000 .

[35]  J. D. Schaffer,et al.  Some experiments in machine learning using vector evaluated genetic algorithms (artificial intelligence, optimization, adaptation, pattern recognition) , 1984 .

[36]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[37]  Peter J. Fleming,et al.  Genetic Algorithms for Multiobjective Optimization: FormulationDiscussion and Generalization , 1993, ICGA.

[38]  Vincenzo Simeone,et al.  Rainfall decreasing and groundwater resources pauperizing in Southern Italy. , 2004 .

[39]  Orazio Giustolisi,et al.  Improving generalization of artificial neural networks in rainfall–runoff modelling / Amélioration de la généralisation de réseaux de neurones artificiels pour la modélisation pluie-débit , 2005 .

[40]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[41]  Roland K. Price,et al.  Managing uncertainty in hydrological models using complementary models , 2003 .

[42]  Jason Smith,et al.  Neural-Network Models of Rainfall-Runoff Process , 1995 .

[43]  null null,et al.  Artificial Neural Networks in Hydrology. II: Hydrologic Applications , 2000 .

[44]  Orazio Giustolisi.  Input–output dynamic neural networks simulating inflow–outflow phenomena in an urban hydrological basin , 2000 .

[45]  Orazio Giustolisi.  Sparse solution in training artificial neural networks , 2004, Neurocomputing.