Global Sensitivity Analysis Approach for Input Selection and System Identification Purposes—A New Framework for Feedforward Neural Networks

A new algorithm for the selection of input variables of neural network is proposed. This new method, applied after the training stage, ranks the inputs according to their importance in the variance of the model output. The use of a global sensitivity analysis technique, extended Fourier amplitude sensitivity test, gives the total sensitivity index for each variable, which allows for the ranking and the removal of the less relevant inputs. Applied to some benchmarking problems in the field of features selection, the proposed approach shows good agreement in keeping the relevant variables. This new method is a useful tool for removing superfluous inputs and for system identification.

[1]  Derek A. Linkens,et al.  Input selection and partition validation for fuzzy modelling using neural network , 1999, Fuzzy Sets Syst..

[2]  N. D. Rao,et al.  Artificial neural networks and their applications to power systems—a bibliographical survey , 1993 .

[3]  Josep M. Sopena,et al.  Performing Feature Selection With Multilayer Perceptrons , 2008, IEEE Transactions on Neural Networks.

[4]  Lennart Ljung,et al.  Nonlinear black-box modeling in system identification: a unified overview , 1995, Autom..

[5]  Alfred Jean Philippe Lauret,et al.  A node pruning algorithm based on a Fourier amplitude sensitivity test method , 2006, IEEE Transactions on Neural Networks.

[6]  Mihir Sen,et al.  Simulation of heat exchanger performance by artificial neural networks , 1999 .

[7]  Rodney L. McClain,et al.  Neural network analysis of fin-tube refrigerating heat exchanger with limited experimental data , 2001 .

[8]  Stefano Tarantola,et al.  A Quantitative Model-Independent Method for Global Sensitivity Analysis of Model Output , 1999, Technometrics.

[9]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[10]  Kenneth W. Bauer,et al.  Determining input features for multilayer perceptrons , 1995, Neurocomputing.

[11]  Yann LeCun,et al.  Optimal Brain Damage , 1989, NIPS.

[12]  J. van Leeuwen,et al.  Neural Networks: Tricks of the Trade , 2002, Lecture Notes in Computer Science.

[13]  Babak Hassibi,et al.  Second Order Derivatives for Network Pruning: Optimal Brain Surgeon , 1992, NIPS.

[14]  Kezhi Mao,et al.  Fast orthogonal forward selection algorithm for feature subset selection , 2002, IEEE Trans. Neural Networks.

[15]  Mikko Lehtokangas.  Modified constructive backpropagation for regression , 2000, Neurocomputing.

[16]  Léon Personnaz,et al.  Nonlinear internal model control using neural networks: application to processes with delay and design issues , 2000, IEEE Trans. Neural Networks Learn. Syst..

[17]  Sitakanta Mohanty,et al.  Sensitivity analysis of a complex, proposed geologic waste disposal system using the Fourier Amplitude Sensitivity Test method , 2001, Reliab. Eng. Syst. Saf..

[18]  Ian T. Nabney,et al.  Netlab: Algorithms for Pattern Recognition , 2002 .

[19]  Klaus-Robert Müller,et al.  Efficient BackProp , 2012, Neural Networks: Tricks of the Trade.

[20]  Gérard Dreyfus,et al.  Ranking a Random Feature for Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[21]  Joseph A. C. Delaney.  Sensitivity analysis , 2018, The African Continental Free Trade Area: Economic and Distributional Effects.

[22]  Richard D. Braatz,et al.  On the "Identification and control of dynamical systems using neural networks" , 1997, IEEE Trans. Neural Networks.

[23]  Russell Reed,et al.  Pruning algorithms-a survey , 1993, IEEE Trans. Neural Networks.

[24]  Jacek M. Zurada,et al.  Perturbation method for deleting redundant inputs of perceptron networks , 1997, Neurocomputing.

[25]  Bin Wang,et al.  ELITE: Ensemble of Optimal Input-Pruned Neural Networks Using TRUST-TECH , 2011, IEEE Transactions on Neural Networks.

[26]  Jian-Bo Yang,et al.  Feature Selection Using Probabilistic Prediction of Support Vector Regression , 2011, IEEE Transactions on Neural Networks.

[27]  Gérard Bloch,et al.  Reduced-Size Kernel Models for Nonlinear Hybrid System Identification , 2011, IEEE Transactions on Neural Networks.

[28]  Nikhil R. Pal,et al.  Selecting Useful Groups of Features in a Connectionist Framework , 2008, IEEE Transactions on Neural Networks.

[29]  Gustavo Deco,et al.  Two Strategies to Avoid Overfitting in Feedforward Networks , 1997, Neural Networks.

[30]  Andries Petrus Engelbrecht,et al.  A new pruning heuristic based on variance analysis of sensitivity information , 2001, IEEE Trans. Neural Networks.

[31]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[32]  Jian-Bo Yang,et al.  Feature Selection for MLP Neural Network: The Use of Random Permutation of Probabilistic Outputs , 2009, IEEE Transactions on Neural Networks.

[33]  Jacek M. Zurada,et al.  Normalized Mutual Information Feature Selection , 2009, IEEE Transactions on Neural Networks.

[34]  Sheng Chen,et al.  Orthogonal least squares methods and their application to non-linear system identification , 1989 .

[35]  David Surry,et al.  Prediction of pressure coefficients on roofs of low buildings using artificial neural networks , 2003 .

[36]  David J. C. MacKay,et al.  A Practical Bayesian Framework for Backpropagation Networks , 1992, Neural Computation.

[37]  Niels Kjølstad Poulsen,et al.  Neural Networks for Modelling and Control of Dynamic Systems: A Practitioner’s Handbook , 2000 .

[38]  Niels Kjølstad Poulsen,et al.  NNSYSID and NNCTRL - MATLAB Tools for System Identification and Control with Neural Networks , 1997 .

[39]  Zerouak Hamza.  Applications of artificial neural-networks for energy systems , 2000 .

[40]  Soteris A. Kalogirou,et al.  Predicting the pressure coefficients in a naturally ventilated test room using artificial neural networks , 2003 .

[41]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.