International Journal of Intelligent Computing and Cybernetics Regularized machine learning through constraint swarm and evolutionary computation applied to regression problems

Purpose – The purpose of this paper is to demonstrate the applicability of swarm and evolutionary techniques for regularized machine learning. Generally, by defining a proper penalty function, regularization laws are embedded into the structure of common least square solutions to increase the numerical stability, sparsity, accuracy and robustness of regression weights. Several regularization techniques have been proposed so far which have their own advantages and disadvantages. Several efforts have been made to find fast and accurate deterministic solvers to handle those regularization techniques. However, the proposed numerical and deterministic approaches need certain knowledge of mathematical programming, and also do not guarantee the global optimality of the obtained solution. In this research, the authors propose the use of constraint swarm and evolutionary techniques to cope with demanding requirements of regularized extreme learning machine (ELM). Design/methodology/approach – To implement the requ...

[1]  A. Kai Qin,et al.  Evolutionary extreme learning machine , 2005, Pattern Recognit..

[2]  Dianhui Wang,et al.  Extreme learning machines: a survey , 2011, Int. J. Mach. Learn. Cybern..

[3]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[4]  Slawomir Zak,et al.  Firefly Algorithm for Continuous Constrained Optimization Tasks , 2009, ICCCI.

[5]  Saeed Behzadipour,et al.  An evolvable self-organizing neuro-fuzzy multilayered classifier with group method data handling and grammar-based bio-inspired supervisors for fault diagnosis of hydraulic systems , 2014, Int. J. Intell. Comput. Cybern..

[6]  Marzuki Khalid,et al.  Structure optimization of neural network for dynamic system modeling using multi-objective genetic algorithm , 2011, Neural Computing and Applications.

[7]  Silvia Curteanu,et al.  Multi-objective optimization of a stacked neural network using an evolutionary hyper-heuristic , 2012, Appl. Soft Comput..

[8]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[9]  Gary G. Yen,et al.  A Self Adaptive Penalty Function Based Algorithm for Constrained Optimization , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[10]  Mark W. Schmidt,et al.  Least Squares Optimization with L 1-Norm Regularization , 2005 .

[11]  Amaury Lendasse,et al.  Regularized extreme learning machine for regression with missing data , 2013, Neurocomputing.

[12]  Lei Chen,et al.  Enhanced random search based incremental extreme learning machine , 2008, Neurocomputing.

[13]  Nasser L. Azad,et al.  Optimally pruned extreme learning machine with ensemble of regularization techniques and negative correlation penalty applied to automotive engine coldstart hydrocarbon emission identification , 2014, Neurocomputing.

[14]  Xin Yao,et al.  A new evolutionary system for evolving artificial neural networks , 1997, IEEE Trans. Neural Networks.

[15]  Xin Yao,et al.  Evolutionary ensembles with negative correlation learning , 2000, IEEE Trans. Evol. Comput..

[16]  Amir Hossein Gandomi,et al.  Evolutionary boundary constraint handling scheme , 2012, Neural Computing and Applications.

[17]  Biruk G. Tessema,et al.  A self-adaptive genetic algorithm for constrained optimization , 2006 .

[18]  Ahmad Mozaffari,et al.  Identification of a dynamic model for shape memory alloy actuator using Hammerstein-Wiener gray box and mutable smart bee algorithm , 2013, Int. J. Intell. Comput. Cybern..

[19]  Pannag R. Sanketi,et al.  Coldstart modeling and optimal control design for automotive SI engines , 2009 .

[20]  Ahmad Mozaffari,et al.  Optimal design of classic Atkinson engine with dynamic specific heat using adaptive neuro-fuzzy inference system and mutable smart bee algorithm , 2013, Swarm Evol. Comput..

[21]  José David Martín-Guerrero,et al.  Regularized extreme learning machine for regression problems , 2011, Neurocomputing.

[22]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[23]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[24]  Amaury Lendasse,et al.  TROP-ELM: A double-regularized ELM using LARS and Tikhonov regularization , 2011, Neurocomputing.

[25]  Ponnuthurai N. Suganthan,et al.  Real-parameter evolutionary multimodal optimization - A survey of the state-of-the-art , 2011, Swarm Evol. Comput..

[26]  Amaury Lendasse,et al.  Extreme Learning Machine: A Robust Modeling Technique? Yes! , 2013, IWANN.

[27]  Nasser L. Azad,et al.  Determining Model Accuracy Requirements for Automotive Engine Coldstart Hydrocarbon Emissions Control , 2012 .

[28]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[29]  Jong-Bae Park,et al.  An Improved Particle Swarm Optimization for Nonconvex Economic Dispatch Problems , 2010 .

[30]  Ahmad Mozaffari,et al.  Vector optimization of laser solid freeform fabrication system using a hierarchical mutable smart bee-fuzzy inference system and hybrid NSGA-II/self-organizing map , 2014, J. Intell. Manuf..

[31]  Jong-Bae Park,et al.  An Improved Particle Swarm Optimization for , 2010 .

[32]  Carlos A. Coello Coello,et al.  Constraint-handling in nature-inspired numerical optimization: Past, present and future , 2011, Swarm Evol. Comput..

[33]  Amaury Lendasse,et al.  OP-ELM: Optimally Pruned Extreme Learning Machine , 2010, IEEE Transactions on Neural Networks.

[34]  D. Karaboga,et al.  On the performance of artificial bee colony (ABC) algorithm , 2008, Appl. Soft Comput..

[35]  Ahmad Mozaffari,et al.  Modeling a shape memory alloy actuator using an evolvable recursive black-box and hybrid heuristic algorithms inspired based on the annual migration of salmons in nature , 2014, Appl. Soft Comput..

[36]  Fei Han,et al.  An improved evolutionary extreme learning machine based on particle swarm optimization , 2013, Neurocomputing.