Stable Forecasting of Environmental Time Series via Long Short Term Memory Recurrent Neural Network

In a recent decade, deep neural networks have been applied for many research areas after achieving dramatic improvements of accuracy in solving complex problems in vision and computational linguistics area. However, some problems, such as environmental modeling, are still limited to benefit from the deep networks because of its difficulty in collecting sufficient data of learning process. In this paper, aside from the accuracy issue, we raise another property—stability—of the deep networks useful for even such data-limited problems, especially in time-series modeling. Recurrent neural networks with memory cell structures, a deep network, can be deemed as a more robust network structure for long-term forecasting under coarse data observation and associated uncertainties, including missing values and sampling/measurement errors. The stability in forecasting is induced from balancing impact of inputs over all time steps in the networks. To analyze this property in various problem conditions, we adapt the recurrent networks with memory structure to environmental time-series problems, such as forecasting water pollution, air pollution, and ozone alarm. In the results, the recurrent networks with memory showed better performance of forecasting in non-stationary environment and long-term time lags.

[1]  R. Burnett,et al.  Lung cancer, cardiopulmonary mortality, and long-term exposure to fine particulate air pollution. , 2002, JAMA.

[2]  P. Kanaroglou,et al.  Mapping real-time air pollution health risk for environmental management: Combining mobile and stationary air pollution monitoring with neural network models. , 2016, Journal of environmental management.

[3]  M. C. Hubbard,et al.  A Comparison of Nonlinear Regression and Neural Network Models for Ground-Level Ozone Forecasting , 2000, Journal of the Air & Waste Management Association.

[4]  Wuxing Wan,et al.  Evidence of widespread ozone-induced visible injury on plants in Beijing, China. , 2014, Environmental pollution.

[5]  Peter Goethals,et al.  Optimisation of Artificial Neural Network (ANN) model design for prediction of macroinvertebrate communities in the Zwalm river basin (Flanders, Belgium) , 2002 .

[6]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[7]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[8]  D. Dockery,et al.  An association between air pollution and mortality in six U.S. cities. , 1993, The New England journal of medicine.

[9]  Kun Zhang,et al.  Forecasting Skewed Biased Stochastic Ozone Days: Analyses and Solutions , 2006, Sixth International Conference on Data Mining (ICDM'06).

[10]  Robert I. McKay,et al.  Machine Learning for Predictive Management: Short and Long term Prediction of Phytoplankton Biomass using Genetic Algorithm Based Recurrent Neural Networks , 2012 .

[11]  Qing Zhang,et al.  Forecasting raw-water quality parameters for the North Saskatchewan River by neural network modeling , 1997 .

[12]  Antonella Zanobetti,et al.  The effect of ozone and PM10 on hospital admissions for pneumonia and chronic obstructive pulmonary disease: a national multicity study. , 2006, American journal of epidemiology.

[13]  M. Huijbregts,et al.  Characterization factors for global warming in life cycle assessment based on damages to humans and ecosystems. , 2009, Environmental science & technology.

[14]  H. Nagy,et al.  Prediction of Sediment Load Concentration in Rivers using Artificial Neural Network Model , 2002 .

[15]  K. Jeong,et al.  Non-linear autoregressive modelling by Temporal Recurrent Neural Networks for the prediction of freshwater phytoplankton dynamics , 2008 .

[16]  Quoc V. Le,et al.  Addressing the Rare Word Problem in Neural Machine Translation , 2014, ACL.

[17]  R. Vinayakumar,et al.  DeepAirNet: Applying Recurrent Networks for Air Quality Prediction , 2018 .

[18]  B. Moss,et al.  Ecology of fresh waters : man and medium, past to future , 1998 .

[19]  Friedrich Recknagel,et al.  Prediction and elucidation of phytoplankton dynamics in the Nakdong River (Korea) by means of a recurrent artificial neural network , 2001 .

[20]  V. Prybutok,et al.  A neural network model forecasting for prediction of daily maximum ozone concentration in an industrialized urban area. , 1996, Environmental pollution.

[21]  Matthew D. Zeiler.  ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[22]  Yoshua Bengio,et al.  On the Properties of Neural Machine Translation: Encoder–Decoder Approaches , 2014, SSST@EMNLP.

[23]  Pedro G. Lind,et al.  Neural network forecast of daily pollution concentration using optimal meteorological data at synoptic and local scales , 2015 .

[24]  Wenrui Huang,et al.  Neural network modeling of salinity variation in Apalachicola River. , 2002, Water research.

[25]  Nguyen Xuan Hoai,et al.  Ecological application of evolutionary computation: Improving water quality forecasts for the Nakdong River, Korea , 2010, IEEE Congress on Evolutionary Computation.

[26]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[27]  Simon M. Mitrovic,et al.  Modelling suppression of cyanobacterial blooms by flow management in a lowland river , 2006 .

[28]  E. Massera,et al.  On field calibration of an electronic nose for benzene estimation in an urban pollution monitoring scenario , 2008 .

[29]  Paulin Coulibaly,et al.  Ground-level ozone forecasting using data-driven methods , 2008 .

[30]  Yu Zhang,et al.  Simple Recurrent Units for Highly Parallelizable Recurrence , 2017, EMNLP.

[31]  George Kurian,et al.  Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation , 2016, ArXiv.

[32]  Kun Zhang,et al.  Forecasting skewed biased stochastic ozone days: analyses, solutions and beyond , 2008, Knowledge and Information Systems.

[33]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[34]  Holger R. Maier,et al.  Flow management strategies to control blooms of the cyanobacterium, Anabaena circinalis, in the River Murray at Morgan, south Australia , 2001 .

[35]  Andrew R.G. Large,et al.  Rehabilitation of River Margins , 1994 .

[36]  S. Vitabile,et al.  Two-days ahead prediction of daily maximum concentrations of SO2, O3, PM10, NO2, CO in the urban area of Palermo, Italy , 2007 .

[37]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[38]  G J Joo,et al.  Modelling community changes of cyanobacteria in a flow regulated river (the lower Nakdong River, S. Korea) by means of a Self-Organizing Map (SOM) , 2005 .

[39]  S. D. Vito,et al.  CO, NO2 and NOx urban pollution monitoring with on-field calibrated electronic nose by automatic bayesian regularization , 2009 .

[40]  Mahad Baawain,et al.  Systematic Approach for the Prediction of Ground-Level Air Pollution (around an Industrial Port) Using an Artificial Neural Network , 2014 .