A hybrid variable selection algorithm for multi-layer perceptron with nonnegative garrote and extremal optimization

In the paper, a new hybrid variable selection algorithm for nonlinear regression multi-layer perceptron (MLP) is proposed. The proposed algorithm applies nonnegative garrote (NNG) to compress the input weights of the MLP. The zero input weights dependent variables will be removed from the initial dataset. Next, a further variable selection is carried out by extremal optimization (EO) algorithm. The new variable selection algorithm integrates powerful global selection ability of NNG and accurate local search ability of EO. Finally, two examples of artificial data sets and an industrial application for a debutanizer column are implemented to demonstrate the performance of the new algorithm. The simulation result demonstrates that the developed algorithm presents d better model performance along with less input variable selected than other state-of-art variable selection methods.

[1]  Tom Heskes,et al.  Massively-parallel best subset selection for ordinary least-squares regression , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).

[2]  Epaminondas Sidiropoulos.  Spatial resource allocation via extremal optimization enhanced by cell-based local search , 2015, Int. J. Model. Simul. Sci. Comput..

[3]  Mohammad R. Akbarzadeh-Totonchi,et al.  Control of elastic joint robot based on electromyogram signal by pre-trained Multi-Layer Perceptron , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[4]  Marcus Randall,et al.  Extremal Optimisation with a Penalty Approach for the Multidimensional Knapsack Problem , 2008, SEAL.

[5]  Patrice Wira,et al.  A new approach based on a linear Multi-Layer Perceptron for identifying on-line harmonics , 2013, IECON 2013 - 39th Annual Conference of the IEEE Industrial Electronics Society.

[6]  Jin Guodong,et al.  The use of Non-Negative Garrote for nonlinear system identification of brushless motors , 2016, 2016 35th Chinese Control Conference (CCC).

[7]  Pudi Sekhar,et al.  An online power system static security assessment module using multi-layer perceptron and radial basis function network , 2016 .

[8]  Xia Li,et al.  A novel particle swarm optimizer hybridized with extremal optimization , 2010, Appl. Soft Comput..

[9]  Vera Chung,et al.  Forecasting wind power in the Mai Liao Wind Farm based on the multi-layer perceptron artificial neural network model with improved simplified swarm optimization , 2014 .

[10]  Jialin Liu,et al.  Development of a variable selection method for soft sensor using artificial neural network and nonnegative garrote , 2014 .

[11]  Jian Liu,et al.  Extremal Optimization with Local Search for the Circular Packing Problem , 2007, Third International Conference on Natural Computation (ICNC 2007).

[12]  David Shan-Hill Wong,et al.  Soft-sensor development with adaptive variable selection using nonnegative garrote , 2013 .

[13]  Stefan Boettcher,et al.  Evolutionary Dynamics of Extremal Optimization , 2009, LION.

[14]  L. Breiman.  Better subset regression using the nonnegative garrote , 1995 .

[15]  R. Haftka,et al.  Multiple surrogates: how cross-validation errors can help us to obtain the best predictor , 2009 .

[16]  Josep M. Sopena,et al.  Performing Feature Selection With Multilayer Perceptrons , 2008, IEEE Transactions on Neural Networks.

[17]  Geoffrey Poitras,et al.  Estimation of the optimal hedge ratio, expected utility, and ordinary least squares regression , 1991 .

[18]  Jafar Sadeghi,et al.  Data-driven soft sensor approach for online quality prediction using state dependent parameter models , 2017 .

[19]  Jorge Nocedal,et al.  An interior algorithm for nonlinear optimization that combines line search and trust region steps , 2006, Math. Program..

[20]  Manabu Kano,et al.  Nearest Correlation-Based Input Variable Weighting for Soft-Sensor Design , 2018, Front. Chem..

[21]  David Shan-Hill Wong,et al.  An inferential modeling method using enumerative PLS based nonnegative garrote regression , 2012 .

[22]  Stefan Boettcher.  Extremal Optimization: Heuristics Via Co-Evolutionary Avalanches , 2000, Comput. Sci. Eng..

[23]  Luigi Fortuna,et al.  Soft sensors for product quality monitoring in debutanizer distillation columns , 2005 .