Sensitivity based selection of inputs and delays for NARX models

In this paper, an extension of sensitivity based pruning (SBP) method for Nonlinear AutoRegressive models with eXogenous inputs (NARX) model is presented. Besides the inputs, input and output delays are simultaneously pruned in terms of the backward elimination. The concept is based on replacement of some regressors by their mean value, which corresponds to the removal of influence of the particular regressors from the network. The method is demonstrated on two datasets. Firstly, one artificial generator is used to test if the method is able to find an optimal set of inputs and delays. Further, the method is used for prediction of gas consumption of a simulated heating for an office building. It is shown that the SBP significantly reduces the complexity of the NARX network without any significant performance degradation. Moreover, it is hypothesized than SBP can be more important for NARX than for simple feedforward neural network, because NARX is more prone to overfitting and has problems with stability.

[1]  Wei Sun,et al.  Application of Neural Network Model Based on Combination of Fuzzy Classification and Input Selection in Short Term Load Forecasting , 2006, 2006 International Conference on Machine Learning and Cybernetics.

[2]  O. Nelles.  Nonlinear System Identification: From Classical Approaches to Neural Networks and Fuzzy Models , 2000 .

[3]  António E. Ruano,et al.  Neural networks based predictive control for thermal comfort and energy savings in public buildings , 2012 .

[4]  I. J. Leontaritis,et al.  Input-output parametric models for non-linear systems Part II: stochastic non-linear systems , 1985 .

[5]  Hava T. Siegelmann,et al.  Computational capabilities of recurrent NARX neural networks , 1997, IEEE Trans. Syst. Man Cybern. Part B.

[6]  Gheith A. Abandah,et al.  Recognizing handwritten Arabic words using grapheme segmentation and recurrent neural networks , 2014, International Journal on Document Analysis and Recognition (IJDAR).

[7]  Kenneth W. Bauer,et al.  Input Feature Selection For Automatic Target Recognition of Temporal Data , 2005 .

[8]  de Mh Martin Wit,et al.  Hambase : heat, air and moisture model for building and systems evaluation , 2006 .

[9]  Christian Endisch,et al.  Optimal Brain Surgeon for General Dynamic Neural Networks , 2007, EPIA Workshops.

[10]  Fiorella Lauro,et al.  Sensitivity Based Feature Selection for Recurrent Neural Network Applied to Forecasting of Heating Gas Consumption , 2014, SOCO-CISIS-ICEUTE.

[11]  Gregory J. Wolff,et al.  Optimal Brain Surgeon and general network pruning , 1993, IEEE International Conference on Neural Networks.

[12]  Kenneth W. Bauer,et al.  Feature screening using signal-to-noise ratios , 2000, Neurocomputing.

[13]  Mansour Sheikhan.  Generation of suprasegmental information for speech using a recurrent neural network and binary gravitational search algorithm for feature selection , 2013, Applied Intelligence.

[14]  Sun-Yuan Kung,et al.  The past is important: a method for determining memory structure in NARX neural networks , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[15]  John E. Moody,et al.  The Effective Number of Parameters: An Analysis of Generalization and Regularization in Nonlinear Learning Systems , 1991, NIPS.