Comparing various artificial neural network types for water temperature prediction in rivers

Summary A number of methods have been proposed for the prediction of streamwater temperature based on various meteorological and hydrological variables. The present study shows a comparison of few types of data-driven neural networks (multi-layer perceptron, product-units, adaptive-network-based fuzzy inference systems and wavelet neural networks) and nearest neighbour approach for short time streamwater temperature predictions in two natural catchments (mountainous and lowland) located in temperate climate zone, with snowy winters and hot summers. To allow wide applicability of such models, autoregressive inputs are not used and only easily available measurements are considered. Each neural network type is calibrated independently 100 times and the mean, median and standard deviation of the results are used for the comparison. Finally, the ensemble aggregation approach is tested. The results show that simple and popular multi-layer perceptron neural networks are in most cases not outperformed by more complex and advanced models. The choice of neural network is dependent on the way the models are compared. This may be a warning for anyone who wish to promote own models, that their superiority should be verified in different ways. The best results are obtained when mean, maximum and minimum daily air temperatures from the previous days are used as inputs, together with the current runoff and declination of the Sun from two recent days. The ensemble aggregation approach allows reducing the mean square error up to several percent, depending on the case, and noticeably diminishes differences in modelling performance obtained by various neural network types.

[1]  Adam P. Piotrowski,et al.  Differential Evolution algorithms applied to Neural Network training suffer from stagnation , 2014, Appl. Soft Comput..

[2]  Ingrid Daubechies,et al.  The wavelet transform, time-frequency localization and signal analysis , 1990, IEEE Trans. Inf. Theory.

[3]  Taha B. M. J. Ouarda,et al.  Diagnostic study and modeling of the annual positive water temperature onset. , 2009 .

[4]  D. Hannah,et al.  Recent advances in stream and river temperature research , 2008 .

[5]  Jaroslaw J. Napiorkowski,et al.  Stream temperature forecasting by means of ensemble of neural networks: Importance of input variables and ensemble size , 2014 .

[6]  Asaad Y. Shamseldin,et al.  Comparative study of different wavelet based neural network models for rainfall–runoff modeling , 2014 .

[7]  Pablo M. Granitto,et al.  Neural network ensembles: evaluation of aggregation algorithms , 2005, Artif. Intell..

[8]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[9]  Jan Adamowski,et al.  Determining the amplitude and timing of streamflow discontinuities: A cross wavelet analysis approach , 2014 .

[10]  Robin T. Clarke,et al.  On the (mis)use of statistical methods in hydro-climatological research , 2010 .

[11]  M. Butts,et al.  Integrated flow and temperature modeling at the catchment scale , 2013 .

[12]  Xin Yao,et al.  A constructive algorithm for training cooperative neural network ensembles , 2003, IEEE Trans. Neural Networks.

[13]  T. Kinouchi,et al.  Analysis of stream temperature and heat budget in an urban river under strong anthropogenic influences , 2013 .

[14]  T. Mayer.  Controls of summer stream temperature in the Pacific Northwest , 2012 .

[15]  Fangfang Liu,et al.  Aquatic metabolism response to the hydrologic alteration in the Yellow River estuary, China , 2015 .

[16]  Asaad Y. Shamseldin,et al.  Comparison of different forms of the Multi-layer Feed-Forward Neural Network method used for river flow forecasting , 2002 .

[17]  O. Kisi,et al.  Suspended sediment modeling using genetic programming and soft computing techniques , 2012 .

[18]  Adam P. Piotrowski,et al.  Product-Units neural networks for catchment runoff forecasting , 2012 .

[19]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[20]  Yu-Bin Yang,et al.  Lung cancer cell identification based on artificial neural network ensembles , 2002, Artif. Intell. Medicine.

[21]  Durdu Ömer Faruk.  A hybrid neural network and ARIMA model for water quality time series prediction , 2010, Eng. Appl. Artif. Intell..

[22]  Niyazi Kilic,et al.  Estimation of stream temperature in firtina creek (Rize-Turkiye) using artificial neural network model. , 2007, Journal of environmental biology.

[23]  Daniel Caissie,et al.  Stream temperature modelling using artificial neural networks: application on Catamaran Brook, New Brunswick, Canada , 2008 .

[24]  Ratko Grbic,et al.  Stream water temperature prediction based on Gaussian process regression , 2013, Expert Syst. Appl..

[25]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[27]  André St-Hilaire,et al.  Streambed temperature dynamics and corresponding heat fluxes in small streams experiencing seasonal ice cover , 2014 .

[28]  R. MacDonald,et al.  Catchment-scale stream temperature response to land disturbance by wildfire governed by surface–subsurface energy exchange and atmospheric controls , 2014 .

[29]  G. Sahoo,et al.  Forecasting stream water temperature using regression analysis, artificial neural network, and chaotic non-linear dynamic models , 2009 .

[30]  Marijana Hadzima-Nyarko,et al.  Implementation of Artificial Neural Networks in Modeling the Water-Air Temperature Relationship of the River Drava , 2014, Water Resources Management.

[31]  G. Poole,et al.  An Ecological Perspective on In-Stream Temperature: Natural Heat Dynamics and Mechanisms of Human-CausedThermal Degradation , 2001, Environmental management.

[32]  P. Coulibaly,et al.  Two decades of anarchy? Emerging themes and outstanding challenges for neural network river forecasting , 2012 .

[33]  Matthias Schmid,et al.  Developing and testing temperature models for regulated systems: A case study on the Upper Delaware River , 2014 .

[34]  X. Wen,et al.  A comparative study of artificial neural network, adaptive neuro fuzzy inference system and support vector machine for forecasting river flow in the semiarid mountain region , 2014 .

[35]  Lee E. Brown,et al.  Flow regulation alters alpine river thermal regimes , 2012 .

[36]  Martin F. Lambert,et al.  Bayesian training of artificial neural networks used for water resources modeling , 2005 .

[37]  Luis Deban,et al.  Assessment of seasonal and polluting effects on the quality of river water by exploratory data analysis , 1998 .

[38]  Marijana Hadzima-Nyarko,et al.  Modelling river temperature from air temperature: case of the River Drava (Croatia) , 2015 .

[39]  C. Zhang,et al.  Convergence of BP algorithm for product unit neural networks with exponential weights , 2008, Neurocomputing.

[40]  Gregory W. Corder,et al.  Nonparametric Statistics : A Step-by-Step Approach , 2014 .

[41]  R. Maheswaran,et al.  Wavelet–Volterra coupled model for monthly stream flow forecasting , 2012 .

[42]  Paulin Coulibaly,et al.  A recurrent neural networks approach using indices of low-frequency climatic variability to forecast regional annual runoff. , 2000 .

[43]  Rao S. Govindaraju,et al.  Prediction of watershed runoff using Bayesian concepts and modular neural networks , 2000 .

[44]  Heinz G. Stefan,et al.  Stream temperature/air temperature relationship : a physical interpretation , 1999 .

[45]  Khulood AlYahya,et al.  Artificial Bee Colony training of neural networks: comparison with back-propagation , 2014, Memetic Comput..

[46]  Achilleas Zapranis,et al.  Wavelet Neural Networks: A Practical Guide , 2011, Neural Networks.

[47]  Jianxun He,et al.  Prediction of event-based stormwater runoff quantity and quality by ANNs developed using PMI-based input selection , 2011 .

[48]  Pavel Kabat,et al.  Global river temperatures and sensitivity to atmospheric warming and changes in river flow , 2011 .

[49]  Petri Koistinen,et al.  Using additive noise in back-propagation training , 1992, IEEE Trans. Neural Networks.

[50]  Wensheng Wang,et al.  Wavelet Network Model and Its Application to the Prediction of Hydrology , 2003 .

[51]  D. Hannah,et al.  Water temperature dynamics in High Arctic river basins , 2011 .

[52]  K. P. Sudheer,et al.  Methods used for the development of neural networks for the prediction of water resource variables in river systems: Current status and future directions , 2010, Environ. Model. Softw..

[53]  Jun Zheng,et al.  Predicting software reliability with neural network ensembles , 2009, Expert Syst. Appl..

[54]  Eric Gaume,et al.  Over-parameterisation, a major obstacle to the use of artificial neural networks in hydrology? , 2003 .

[55]  Geoffrey E. Hinton,et al.  A general framework for parallel distributed processing , 1986 .

[56]  Chuntian Cheng,et al.  A comparison of performance of several artificial intelligence , 2009 .

[57]  Dorota Mirosław-Świątek,et al.  Scenarios of the spread of a waste heat discharge in a river — Vistula River case study , 2012, Acta Geophysica.

[58]  G. Sahoo,et al.  Use of neural network to predict flash flood and attendant water qualities of a mountainous stream on Oahu, Hawaii , 2006 .

[59]  Adam P. Piotrowski,et al.  Comparing large number of metaheuristics for artificial neural networks training to predict water temperature in a natural river , 2014, Comput. Geosci..

[60]  James F. Frenzel,et al.  Training product unit neural networks with genetic algorithms , 1993, IEEE Expert.

[61]  Alípio Mário Jorge,et al.  Ensemble approaches for regression: A survey , 2012, CSUR.

[62]  Ozgur Kisi,et al.  Applications of hybrid wavelet–Artificial Intelligence models in hydrology: A review , 2014 .

[63]  F. Anctil,et al.  An experiment on the evolution of an ensemble of neural networks for streamflow forecasting , 2009 .

[64]  Jan Adamowski,et al.  Comparison of Multivariate Regression and Artificial Neural Networks for Peak Urban Water-Demand Forecasting: Evaluation of Different ANN Learning Algorithms , 2010 .

[65]  D. Caissie.  The thermal regime of rivers : a review , 2006 .

[66]  Geoffrey E. Hinton,et al.  Simplifying Neural Networks by Soft Weight-Sharing , 1992, Neural Computation.

[67]  Marjan Mernik,et al.  A chess rating system for evolutionary algorithms: A new method for the comparison and ranking of evolutionary algorithms , 2014, Inf. Sci..

[68]  David E. Rumelhart,et al.  Product Units: A Computationally Powerful and Biologically Plausible Extension to Backpropagation Networks , 1989, Neural Computation.

[69]  Leandro Nunes de Castro,et al.  Bio-inspired and gradient-based algorithms to train MLPs: The influence of diversity , 2009, Inf. Sci..

[70]  Richard W. Battarbee,et al.  Detecting changing river temperatures in England and Wales , 2015 .

[71]  Ozgur Kisi,et al.  Modelling of chemical oxygen demand by using ANNs, ANFIS and k-means clustering techniques , 2014 .

[72]  César Hervás-Martínez,et al.  Evolutionary product unit based neural networks for regression , 2006, Neural Networks.

[73]  Christian Blum,et al.  An ant colony optimization algorithm for continuous optimization: application to feed-forward neural network training , 2007, Neural Computing and Applications.

[74]  R. MacDonald,et al.  A process-based stream temperature modelling approach for mountain regions , 2014 .

[75]  J. Adamowski,et al.  A wavelet neural network conjunction model for groundwater level forecasting , 2011 .

[76]  André St-Hilaire,et al.  DEVELOPMENT OF A STOCHASTIC WATER TEMPERATURE MODEL AND PROJECTION OF FUTURE WATER TEMPERATURE AND EXTREME EVENTS IN THE OUELLE RIVER BASIN IN QUÉBEC, CANADA , 2013 .

[77]  Adam P. Piotrowski,et al.  A comparison of methods to avoid overfitting in neural networks training in the case of catchment runoff modelling , 2013 .

[78]  Heinz G. Stefan,et al.  A nonlinear regression model for weekly stream temperatures , 1998 .

[79]  Ozgur Kisi,et al.  A wavelet-support vector machine conjunction model for monthly streamflow forecasting , 2011 .

[80]  Mohammad Bagher Menhaj,et al.  Training feedforward networks with the Marquardt algorithm , 1994, IEEE Trans. Neural Networks.

[81]  H. Prommer,et al.  Elucidating temperature effects on seasonal variations of biogeochemical turnover rates during riverbank filtration , 2012 .

[82]  Tyler Wagner,et al.  A regional neural network ensemble for predicting mean daily river water temperature , 2014 .

[83]  R. Butryn,et al.  Summer Stream Water Temperature Models for Great Lakes Streams: New York , 2010 .

[84]  K. P. Sudheer,et al.  A neuro-fuzzy computing technique for modeling hydrological time series , 2004 .

[85]  Michael Schmitt,et al.  On the Complexity of Computing and Learning with Multiplicative Neural Networks , 2002, Neural Computation.

[86]  Terry Windeatt,et al.  Ensemble Pruning Using Spectral Coefficients , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[87]  Joni-Kristian Kämäräinen,et al.  Differential Evolution Training Algorithm for Feed-Forward Neural Networks , 2003, Neural Processing Letters.

[88]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[89]  Robert J. Abrahart,et al.  Multi-model data fusion for hydrological forecasting , 2001 .

[90]  Carol M. Babyak,et al.  Changes In Stream Temperatures In Response To Restoration Of Groundwater Discharge And Solar Heating In A Culverted, Urban Stream , 2010 .

[91]  Henry C. W. Lau,et al.  A fuzzy multi-criteria decision support procedure for enhancing information delivery in extended enterprise networks , 2003 .

[92]  K. Chau,et al.  Predicting monthly streamflow using data‐driven models coupled with data‐preprocessing techniques , 2009 .

[93]  Dominique Thiéry,et al.  A multimodel comparison for assessing water temperatures under changing climate conditions via the equilibrium temperature concept: case study of the Middle Loire River, France , 2014 .

[94]  Taha B. M. J. Ouarda,et al.  Daily river water temperature forecast model with a k‐nearest neighbour approach , 2012 .

[95]  Clifford I. Voss,et al.  Climate change impacts on the temperature and magnitude of groundwater discharge from shallow, unconfined aquifers , 2014 .

[96]  F. Schmidt.  Statistical Significance Testing and Cumulative Knowledge in Psychology: Implications for Training of Researchers , 1996 .

[97]  Teresa Bernarda Ludermir,et al.  A multi-objective memetic and hybrid methodology for optimizing the parameters and performance of artificial neural networks , 2010, Neurocomputing.

[98]  Chandranath Chatterjee,et al.  Development of an accurate and reliable hourly flood forecasting model using wavelet–bootstrap–ANN (WBANN) hybrid approach , 2010 .

[99]  Z. Kern,et al.  Multidecadal changes in the river ice regime of the lower course of the River Drava since AD 1875 , 2015 .

[100]  Kwok-wing Chau,et al.  A review on integration of artificial intelligence into water quality modelling. , 2006, Marine pollution bulletin.

[101]  E. Maurer,et al.  Effects of climate change on stream temperature, dissolved oxygen, and sediment concentration in the Sierra Nevada in California , 2013 .

[102]  Travis O. Brenden,et al.  A Comparison of Statistical Approaches for Predicting Stream Temperatures Across Heterogeneous Landscapes 1 , 2009 .

[103]  Dimitri Solomatine,et al.  Experimental investigation of the predictive capabilities of data driven modeling techniques in hydrology - Part 2: Application , 2009 .

[104]  Lutz Prechelt,et al.  Automatic early stopping using cross validation: quantifying the criteria , 1998, Neural Networks.

[105]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[106]  Michael Y. Hu,et al.  Forecasting with artificial neural networks: The state of the art , 1997 .

[107]  Robin T. Clarke,et al.  A critique of present procedures used to compare performance of rainfall-runoff models , 2008 .

[108]  Variation of stream temperature among mesoscale habitats within stream reaches: southern Appalachians , 2014 .

[109]  P. Pekárová,et al.  Is the Water Temperature of the Danube River at Bratislava, Slovakia, Rising? , 2008 .

[110]  Martin Mandischer.  A comparison of evolution strategies and backpropagation for neural network training , 2002, Neurocomputing.

[111]  Mysore G. Satish,et al.  Predicting water temperatures using a deterministic model : Application on Miramichi River catchments (New Brunswick, Canada) , 2007 .

[112]  Adam P. Piotrowski,et al.  Comparison of evolutionary computation techniques for noise injected neural network training to estimate longitudinal dispersion coefficients in rivers , 2012, Expert Syst. Appl..

[113]  A. A. Mahboubi,et al.  Impervious surfaces and sewer pipe effects on stormwater runoff temperature , 2013 .