Mutual Information Based Input Variable Selection Algorithm and Wavelet Neural Network for Time Series Prediction

In this paper we have presented an Integrated Wavelet Neural Network (WNN) model and Mutual Information (MI)-based input selection algorithm for time series prediction. Based on MI the proper input variables, which describe the time series' dynamics properly, will be selected. The WNN Prediction model uses selected variables and predicts the future. This model utilized for time series prediction benchmark in NN3 competition and sunspot data. Comprehensive results show that integrated Mutual information based input variable selection algorithm and wavelet network based prediction model, which uses selected variable from lagged value, outperforms other models in prediction of time series.

[1]  Gérard Dreyfus,et al.  Initialization by selection for wavelet network training , 2000, Neurocomputing.

[2]  Jiwen Dong,et al.  Time-series prediction using a local linear wavelet neural network , 2006, Neurocomputing.

[3]  Mohammad Fazle Azeem,et al.  Generalized Wavelet Neural Network Model and its Application in Time Series Prediction , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[4]  Sheng Chen,et al.  Orthogonal least squares methods and their application to non-linear system identification , 1989 .

[5]  Babak Nadjar Araabi,et al.  Mutual Information Based Input Selection in Neuro-Fuzzy Modeling for Short Term Load Forecasting of Iran National Power System , 2007, 2007 IEEE International Conference on Control and Automation.

[6]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Roberto Battiti,et al.  Using mutual information for selecting features in supervised neural net learning , 1994, IEEE Trans. Neural Networks.

[8]  Stephen A. Billings,et al.  A new class of wavelet networks for nonlinear system identification , 2005, IEEE Transactions on Neural Networks.

[9]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[10]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[11]  Amaury Lendasse,et al.  Mutual information and gamma test for input selection , 2005, ESANN.

[12]  Chris H. Q. Ding,et al.  Minimum Redundancy Feature Selection from Microarray Gene Expression Data , 2005, J. Bioinform. Comput. Biol..

[13]  Cyrus Shahabi,et al.  Feature subset selection and feature ranking for multivariate time series , 2005, IEEE Transactions on Knowledge and Data Engineering.

[14]  Caro Lucas,et al.  Input Variables Selection Using Mutual Information for Neuro Fuzzy Modeling with the Application to Time Series Forecasting , 2007, 2007 International Joint Conference on Neural Networks.

[15]  Mohamed A. Deriche,et al.  An optimal feature selection technique using the concept of mutual information , 2001, Proceedings of the Sixth International Symposium on Signal Processing and its Applications (Cat.No.01EX467).

[16]  Qinghua Zhang,et al.  Using wavelet network in nonparametric estimation , 1997, IEEE Trans. Neural Networks.

[17]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .