A comparative study of wavelet-based ANN and classical techniques for geophysical time-series forecasting

Abstract Time-series modeling forms an important area of research in geophysics. Time-series models can be linear, like linear state-space models or non-linear, like artificial neural networks. One way to judge the goodness of different models associated with a given time-series is to assess the prediction capabilities of these models. Some of the important techniques used in time-series forecasting are: (i) Minimum mean squared error (MMSE) forecast obtained using conditional means of ARIMA(p,d,q) models, (ii) Kalman filter approach and (iii) Artificial neural networks (ANN) approach. However, the wavelet-based versions of these techniques, respectively denoted as W-MMSE, W-Kalman and W-ANN, rather than the original techniques themselves, have been found to possess better capabilities in forecasting the highly nonlinear geophysical data. Using the original and predicted data, these observations have been validated by determining the RMSE (root mean squared error) and correlation coefficients between them. The prediction capabilities of both versions of the above techniques are tested on (i) the ionospheric total electron content (TEC) data, (ii) the daily average rainfall data and (iii) the gamma-ray log data from an offshore oil well, off the west coast of India. The TEC and daily average rainfall data sets designate as examples of data with very high correlations pertaining over very large lags. They also have a strong seasonality component associated with them. However, the gamma-ray log data sets show no seasonality component and have no trends associated with them. Therefore, the choice of different nonlinear data sets having diverse sources of their origins are apt to test the forecasting capabilities of these techniques. It has been observed that W-ANN gives the best prediction, when compared with the other algorithms discussed in this paper. This is believed to be due to the use of non-linear activation function by ANNs to produce regression that results in capturing the inherent non-linear dynamics of the process effectively. The results also show the usefulness of discrete wavelet transform (DWT) coefficients as training features for both linear and non-linear forecasting approaches. The better performance of the wavelet-based forecasting algorithms can be attributed to the fact that DWT coefficients are wide-sense stationary sequences. Thus the wavelet-based versions of these models like W-MMSE and W-Kalman provide better fit to these coefficients, compared to the original time-series data itself.

[1]  T.,et al.  Training Feedforward Networks with the Marquardt Algorithm , 2004 .

[2]  E. Chandrasekhar,et al.  Wavelet Analysis of Geophysical Well-log Data of Bombay Offshore Basin, India , 2012, Mathematical Geosciences.

[3]  Vikram M. Gadre,et al.  Construction of Wavelets: Principles and Practices , 2013 .

[4]  Siem Jan Koopman,et al.  A simple and efficient simulation smoother for state space time series analysis , 2002 .

[5]  R. Kálmán.  Mathematical description of linear dynamical systems , 1963 .

[6]  D. Posada,et al.  Model selection and model averaging in phylogenetics: advantages of akaike information criterion and bayesian approaches over likelihood ratio tests. , 2004, Systematic biology.

[7]  Shaohan Hu,et al.  DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing , 2016, WWW.

[8]  Kodjo Agbossou,et al.  Time series prediction using artificial wavelet neural network and multi-resolution analysis: Application to wind speed data , 2016 .

[9]  Warren S. Sarle,et al.  Neural Networks and Statistical Models , 1994 .

[10]  Amy Loutfi,et al.  A review of unsupervised feature learning and deep learning for time-series modeling , 2014, Pattern Recognit. Lett..

[11]  G. Seemala,et al.  Statistics of total electron content depletions observed over the South American continent for the year 2008 , 2011 .

[12]  H. K. Cigizoglu,et al.  Prediction of daily precipitation using wavelet—neural networks , 2009 .

[13]  Robert J. Marks,et al.  Electric load forecasting using an artificial neural network , 1991 .

[14]  Sorin Moga,et al.  A wavelet based prediction method for time series , 2010 .

[15]  A. Walden,et al.  Wavelet Methods for Time Series Analysis , 2000 .

[16]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[17]  S. Mallat.  Multiresolution approximations and wavelet orthonormal bases of L^2(R) , 1989 .

[18]  Michael Y. Hu,et al.  Forecasting with artificial neural networks: The state of the art , 1997 .

[19]  Xiaodong Luo,et al.  An Ensemble 4D Seismic History Matching Framework with Sparse Representation Based on Wavelet Multiresolution Analysis , 2016, 1603.04577.

[20]  Marcus O'Connor,et al.  Artificial neural network models for forecasting and decision making , 1994 .

[21]  R. E. Kalman,et al.  A New Approach to Linear Filtering and Prediction Problems , 2002 .

[22]  William Remus,et al.  Neural Network Models for Time Series Forecasts , 1996 .

[23]  J. L. Holloway.  Smoothing and Filtering of Time Series and Space Fields , 1958 .

[24]  H. Akaike.  Fitting autoregressive models for prediction , 1969 .

[25]  R. V. Sachs,et al.  Wavelets in time-series analysis , 1999, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[26]  Jiwen Dong,et al.  Time-series prediction using a local linear wavelet neural network , 2006, Neurocomputing.

[27]  N. Huang,et al.  The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis , 1998, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[28]  M. G. Kendall,et al.  A Study in the Analysis of Stationary Time-Series. , 1955 .

[29]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[30]  G. Seemala,et al.  Multifractal detrended fluctuation analysis of ionospheric total electron content data during solar minimum and maximum , 2016 .

[31]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[32]  Ilona Weinreich,et al.  Wavelet-based prediction of oil prices , 2005 .

[33]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[34]  Robert Hecht-Nielsen,et al.  Theory of the backpropagation neural network , 1989, International 1989 Joint Conference on Neural Networks.

[35]  Le Zhang,et al.  Ensemble deep learning for regression and time series forecasting , 2014, 2014 IEEE Symposium on Computational Intelligence in Ensemble Learning (CIEL).

[36]  Jürgen Schmidhuber,et al.  Applying LSTM to Time Series Predictable through Time-Window Approaches , 2000, ICANN.

[37]  Donald F. Specht,et al.  A general regression neural network , 1991, IEEE Trans. Neural Networks.

[38]  Melvin J. Hinich,et al.  Time Series Analysis by State Space Methods , 2001 .

[39]  Paulin Coulibaly,et al.  Nonstationary hydrological time series forecasting using nonlinear dynamic methods , 2005 .