Regularized machine learning through constraint swarm and evolutionary computation applied to regression problems

Purpose – The purpose of this paper is to demonstrate the applicability of swarm and evolutionary techniques for regularized machine learning. Generally, by defining a proper penalty function, regularization laws are embedded into the structure of common least square solutions to increase the numerical stability, sparsity, accuracy and robustness of regression weights. Several regularization techniques have been proposed so far which have their own advantages and disadvantages. Several efforts have been made to find fast and accurate deterministic solvers to handle those regularization techniques. However, the proposed numerical and deterministic approaches need certain knowledge of mathematical programming, and also do not guarantee the global optimality of the obtained solution. In this research, the authors propose the use of constraint swarm and evolutionary techniques to cope with demanding requirements of regularized extreme learning machine (ELM). Design/methodology/approach – To implement the requ...

[1]  Cees G. M. Snoek,et al.  Variable Selection , 2019, Model-Based Clustering and Classification for Data Science.

[2]  Ahmad Mozaffari,et al.  Vector optimization of laser solid freeform fabrication system using a hierarchical mutable smart bee-fuzzy inference system and hybrid NSGA-II/self-organizing map , 2014, J. Intell. Manuf..

[3]  Saeed Behzadipour,et al.  An evolvable self-organizing neuro-fuzzy multilayered classifier with group method data handling and grammar-based bio-inspired supervisors for fault diagnosis of hydraulic systems , 2014, Int. J. Intell. Comput. Cybern..

[4]  Nasser L. Azad,et al.  Optimally pruned extreme learning machine with ensemble of regularization techniques and negative correlation penalty applied to automotive engine coldstart hydrocarbon emission identification , 2014, Neurocomputing.

[5]  Ahmad Mozaffari,et al.  Identification of a dynamic model for shape memory alloy actuator using Hammerstein-Wiener gray box and mutable smart bee algorithm , 2013, Int. J. Intell. Comput. Cybern..

[6]  Ahmad Mozaffari,et al.  Optimal design of classic Atkinson engine with dynamic specific heat using adaptive neuro-fuzzy inference system and mutable smart bee algorithm , 2013, Swarm Evol. Comput..

[7]  Fei Han,et al.  An improved evolutionary extreme learning machine based on particle swarm optimization , 2013, Neurocomputing.

[8]  Amaury Lendasse,et al.  Extreme Learning Machine: A Robust Modeling Technique? Yes! , 2013, IWANN.

[9]  Nasser L. Azad,et al.  Determining Model Accuracy Requirements for Automotive Engine Coldstart Hydrocarbon Emissions Control , 2012 .

[10]  A. Gandomi,et al.  Evolutionary boundary constraint handling scheme , 2012, Neural Computing and Applications.

[11]  Carlos A. Coello Coello,et al.  Constraint-handling in nature-inspired numerical optimization: Past, present and future , 2011, Swarm Evol. Comput..

[12]  José David Martín-Guerrero,et al.  Regularized extreme learning machine for regression problems , 2011, Neurocomputing.

[13]  Amaury Lendasse,et al.  TROP-ELM: A double-regularized ELM using LARS and Tikhonov regularization , 2011, Neurocomputing.

[14]  Ponnuthurai N. Suganthan,et al.  Real-parameter evolutionary multimodal optimization - A survey of the state-of-the-art , 2011, Swarm Evol. Comput..

[15]  Dianhui Wang,et al.  Extreme learning machines: a survey , 2011, Int. J. Mach. Learn. Cybern..

[16]  Marzuki Khalid,et al.  Structure optimization of neural network for dynamic system modeling using multi-objective genetic algorithm , 2011, Neural Computing and Applications.

[17]  Jong-Bae Park,et al.  An Improved Particle Swarm Optimization for Nonconvex Economic Dispatch Problems , 2010, IEEE Transactions on Power Systems.

[18]  Lei Chen,et al.  Enhanced random search based incremental extreme learning machine , 2008, Neurocomputing.

[19]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[20]  Gary G. Yen,et al.  A Self Adaptive Penalty Function Based Algorithm for Constrained Optimization , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[21]  A. Kai Qin,et al.  Evolutionary extreme learning machine , 2005, Pattern Recognit..

[22]  Xin Yao,et al.  Evolutionary ensembles with negative correlation learning , 2000, IEEE Trans. Evol. Comput..

[23]  Xin Yao,et al.  A new evolutionary system for evolving artificial neural networks , 1997, IEEE Trans. Neural Networks.

[24]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[25]  Ahmad Mozaffari,et al.  Modeling a shape memory alloy actuator using an evolvable recursive black-box and hybrid heuristic algorithms inspired based on the annual migration of salmons in nature , 2014, Appl. Soft Comput..

[26]  Silvia Curteanu,et al.  Multi-objective optimization of a stacked neural network using an evolutionary hyper-heuristic , 2012, Appl. Soft Comput..

[27]  Amaury Lendasse,et al.  OP-ELM: Optimally Pruned Extreme Learning Machine , 2010, IEEE Transactions on Neural Networks.

[28]  Pannag R. Sanketi,et al.  Coldstart modeling and optimal control design for automotive SI engines , 2009 .

[29]  D. Karaboga,et al.  On the performance of artificial bee colony (ABC) algorithm , 2008, Appl. Soft Comput..

[30]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[31]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.