Generalized versus non-generalized neural network model for multi-lead inflow forecasting at Aswan High Dam

Abstract. Artificial neural networks (ANN) have been found efficient, particularly in problems where characteristics of the processes are stochastic and difficult to describe using explicit mathematical models. However, time series prediction based on ANN algorithms is fundamentally difficult and faces problems. One of the major shortcomings is the search for the optimal input pattern in order to enhance the forecasting capabilities for the output. The second challenge is the over-fitting problem during the training procedure and this occurs when ANN loses its generalization. In this research, autocorrelation and cross correlation analyses are suggested as a method for searching the optimal input pattern. On the other hand, two generalized methods namely, Regularized Neural Network (RNN) and Ensemble Neural Network (ENN) models are developed to overcome the drawbacks of classical ANN models. Using Generalized Neural Network (GNN) helped avoid over-fitting of training data which was observed as a limitation of classical ANN models. Real inflow data collected over the last 130 years at Lake Nasser was used to train, test and validate the proposed model. Results show that the proposed GNN model outperforms non-generalized neural network and conventional auto-regressive models and it could provide accurate inflow forecasting.

[1]  David Chiu,et al.  BOOK REVIEW: "PATTERN CLASSIFICATION", R. O. DUDA, P. E. HART and D. G. STORK, Second Edition , 2001 .

[2]  Bertil Svensson,et al.  Using and Designing Massively Parallel Computers for Artificial Neural Neural Networks , 1992, J. Parallel Distributed Comput..

[3]  Harris Drucker,et al.  Boosting and Other Ensemble Methods , 1994, Neural Computation.

[4]  Chuntian Cheng,et al.  Multiple criteria rainfall–runoff model calibration using a parallel genetic algorithm in a cluster of computers / Calage multi-critères en modélisation pluie–débit par un algorithme génétique parallèle mis en œuvre par une grappe d'ordinateurs , 2005 .

[5]  Chuntian Cheng,et al.  A comparison of performance of several artificial intelligence methods for forecasting monthly discharge time series , 2009 .

[6]  Igor V. Tetko,et al.  Data modelling with neural networks: Advantages and limitations , 1997, J. Comput. Aided Mol. Des..

[7]  Z. Zainuddin,et al.  Improving the Convergence of the Backpropagation Algorithm Using Local Adaptive Techniques , 2007, International Conference on Computational Intelligence.

[8]  Bernard Bobée,et al.  Daily reservoir inflow forecasting using artificial neural networks with stopped training approach , 2000 .

[9]  Arthur Gelb,et al.  Applied Optimal Estimation , 1974 .

[10]  Chidchanok Lursinsap,et al.  Time-series data prediction based on reconstruction of missing samples and selective ensembling of FIR neural networks , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[11]  I. Rodríguez‐Iturbe,et al.  Random Functions and Hydrology , 1984 .

[12]  Chuntian Cheng,et al.  Combining a fuzzy optimal model with a genetic algorithm to solve multi-objective rainfall–runoff model calibration , 2002 .

[13]  Ahmed El-Shafie,et al.  Neural Network Model for Nile River Inflow Forecasting Based on Correlation Analysis of Historical Inflow Data , 2008 .

[14]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[15]  David G. Stork,et al.  Pattern Classification , 1973 .

[16]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[17]  Aris P. Georgakakos,et al.  Decision support systems for integrated water resources management with an application to the nile basin , 2007 .

[18]  John G. Taylor,et al.  Quantifying multivariate classification performance: the problem of overfitting , 1999, Optics & Photonics.

[19]  Peter K. Kitanidis,et al.  Adaptive filtering through detection of isolated transient errors in rainfall‐runoff models , 1980 .

[20]  K. Chau,et al.  Predicting monthly streamflow using data‐driven models coupled with data‐preprocessing techniques , 2009 .

[21]  Lutz Prechelt,et al.  Early Stopping - But When? , 2012, Neural Networks: Tricks of the Trade.

[22]  Amir F. Atiya,et al.  A comparison between neural-network forecasting techniques-case study: river flow forecasting , 1999, IEEE Trans. Neural Networks.

[23]  Maureen Caudill,et al.  Neural network training tips and techniques , 1991 .

[24]  Chuntian Cheng,et al.  Using support vector machines for long-term discharge prediction , 2006 .

[25]  Paulin Coulibaly,et al.  Real-time short-term natural water inflows forecasting using recurrent neural networks , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[26]  Peter K. Kitanidis,et al.  Real‐time forecasting with a conceptual hydrologic model: 1. Analysis of uncertainty , 1980 .

[27]  Kwok-wing Chau,et al.  Particle Swarm Optimization Training Algorithm for ANNs in Stage Prediction of Shing Mun River , 2006 .

[28]  Chao-Lin Chiu,et al.  Applications of Kalman filter to hydrology, hydraulics, and water resources : proceedings of AGU Chapman Conference, held at University of Pittsburgh, Pittsburgh, Pennsylvania, U.S.A., May 22-24, 1978 , 1978 .

[29]  D. Rumelhart,et al.  Predicting sunspots and exchange rates with connectionist networks , 1991 .

[30]  Cullen Schaffer Overfitting avoidance as bias , 2004, Machine Learning.

[31]  E. Todini,et al.  A stable estimator for linear models: 2. Real world hydrologic applications , 1976 .

[32]  Gregg D. Wilensky,et al.  Neural Network Studies , 1993 .

[33]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1972 .

[34]  G. J. Gibson,et al.  On the decision regions of multilayer perceptrons , 1990, Proc. IEEE.

[35]  N. Bodor,et al.  Neural network studies: Part 3. Prediction of partition coefficients , 1994 .

[36]  Ahmed El-Shafie,et al.  A neuro-fuzzy model for inflow forecasting of the Nile river at Aswan high dam , 2007 .

[37]  B. Bobée,et al.  Multivariate Reservoir Inflow Forecasting Using Temporal Neural Networks , 2001 .

[38]  Igor V. Tetko,et al.  Neural network studies, 1. Comparison of overfitting and overtraining , 1995, J. Chem. Inf. Comput. Sci..

[39]  Arjen van Ooyen,et al.  Improving the convergence of the back-propagation algorithm , 1992, Neural Networks.

[40]  Ahmed El-Shafie,et al.  Enhancing Inflow Forecasting Model at Aswan High Dam Utilizing Radial Basis Neural Network and Upstream Monitoring Stations Measurements , 2009 .

[41]  Jun-Ping Zhang,et al.  Time series prediction based on ensemble ANFIS , 2005, 2005 International Conference on Machine Learning and Cybernetics.

[42]  Brian D. Ripley,et al.  Pattern Recognition and Neural Networks , 1996 .