Restricted Boltzmann Machine-Assisted Estimation of Distribution Algorithm for Complex Problems

A novel algorithm, called restricted Boltzmann machine-assisted estimation of distribution algorithm, is proposed for solving computationally expensive optimization problems with discrete variables. First, the individuals are evaluated using expensive fitness functions of the complex problems, and some dominant solutions are selected to construct the surrogate model. The restricted Boltzmann machine (RBM) is built and trained with the dominant solutions to implicitly extract the distributed representative information of the decision variables in the promising subset. The visible layer’s probability of the RBM is designed as the sampling probability model of the estimation of distribution algorithm (EDA) and is updated dynamically along with the update of the dominant subsets. Second, according to the energy function of the RBM, a fitness surrogate is developed to approximate the expensive individual fitness evaluations and participates in the evolutionary process to reduce the computational cost. Finally, model management is developed to train and update the RBM model with newly dominant solutions. A comparison of the proposed algorithm with several state-of-the-art surrogate-assisted evolutionary algorithms demonstrates that the proposed algorithm effectively and efficiently solves complex optimization problems with smaller computational cost.

[1]  Taimoor Akhtar,et al.  Multi objective optimization of computationally expensive multi-modal functions with RBF surrogates and multi-rule selection , 2016, J. Glob. Optim..

[2]  Kaisa Miettinen,et al.  A Surrogate-Assisted Reference Vector Guided Evolutionary Algorithm for Computationally Expensive Many-Objective Optimization , 2018, IEEE Transactions on Evolutionary Computation.

[3]  Xiaoyan Sun,et al.  A New Surrogate-Assisted Interactive Genetic Algorithm With Weighted Semisupervised Learning , 2013, IEEE Transactions on Cybernetics.

[4]  Hani Hagras,et al.  Multiobjective Evolutionary Optimization of Type-2 Fuzzy Rule-Based Systems for Financial Data Classification , 2017, IEEE Transactions on Fuzzy Systems.

[5]  Geoffrey E. Hinton Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[6]  Lok-Won Kim,et al.  DeepX: Deep Learning Accelerator for Restricted Boltzmann Machine Artificial Neural Networks , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[7]  Paul Smolensky,et al.  Information processing in dynamical systems: foundations of harmony theory , 1986 .

[8]  Lamjed Ben Said,et al.  Steady state IBEA assisted by MLP neural networks for expensive multi-objective optimization problems , 2014, GECCO.

[9]  Xin Yao,et al.  Turning High-Dimensional Optimization Into Computationally Expensive Optimization , 2018, IEEE Transactions on Evolutionary Computation.

[10]  Antonio Orlandi Differential Evolutionary Multiple-Objective Sequential Optimization of a Power Delivery Network , 2018, IEEE Transactions on Electromagnetic Compatibility.

[11]  Bum-Joo Lee,et al.  Evolutionary Optimization for Optimal Hopping of Humanoid Robots , 2017, IEEE Transactions on Industrial Electronics.

[12]  Rommel G. Regis,et al.  Evolutionary Programming for High-Dimensional Constrained Expensive Black-Box Optimization Using Radial Basis Functions , 2014, IEEE Transactions on Evolutionary Computation.

[13]  Scott Klasky,et al.  Personalized Search Inspired Fast Interactive Estimation of Distribution Algorithm and Its Application , 2017, IEEE Transactions on Evolutionary Computation.

[14]  Na Lu,et al.  A Deep Learning Scheme for Motor Imagery Classification based on Restricted Boltzmann Machines , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[15]  Chi-Keong Goh,et al.  Multiproblem Surrogates: Transfer Evolutionary Multiobjective Optimization of Computationally Expensive Problems , 2019, IEEE Transactions on Evolutionary Computation.

[16]  Kenneth E. Barner,et al.  Exploiting Restricted Boltzmann Machines and Deep Belief Networks in Compressed Sensing , 2017, IEEE Transactions on Signal Processing.

[17]  Kay Chen Tan,et al.  An Energy-Based Sampling Technique for Multi-Objective Restricted Boltzmann Machine , 2013, IEEE Transactions on Evolutionary Computation.

[18]  Hemant A. Patil,et al.  Novel Unsupervised Auditory Filterbank Learning Using Convolutional RBM for Speech Recognition , 2016, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[19]  Yaochu Jin,et al.  Surrogate-assisted evolutionary computation: Recent advances and future challenges , 2011, Swarm Evol. Comput..

[20]  Jianchao Zeng,et al.  Surrogate-Assisted Cooperative Swarm Optimization of High-Dimensional Expensive Problems , 2017, IEEE Transactions on Evolutionary Computation.

[21]  Ji-Min Kim,et al.  Space-Time Kriging Surrogate Model to Consider Uncertainty of Time Interval of Torque Curve for Electric Power Steering Motor , 2018, IEEE Transactions on Magnetics.

[22]  Qingfu Zhang,et al.  A Gaussian Process Surrogate Model Assisted Evolutionary Algorithm for Medium Scale Expensive Optimization Problems , 2014, IEEE Transactions on Evolutionary Computation.

[23]  Xingyi Zhang,et al.  A Mixed Representation-Based Multiobjective Evolutionary Algorithm for Overlapping Community Detection , 2017, IEEE Transactions on Cybernetics.

[24]  MengChu Zhou,et al.  Optimal Load Scheduling of Plug-In Hybrid Electric Vehicles via Weight-Aggregation Multi-Objective Evolutionary Algorithms , 2017, IEEE Transactions on Intelligent Transportation Systems.

[25]  Ata Kabán,et al.  Toward Large-Scale Continuous EDA: A Random Matrix Theory Perspective , 2013, Evolutionary Computation.

[26]  Zhenkai Zhang,et al.  A novel resource scheduling method of netted radars based on Markov decision process during target tracking in clutter , 2016, EURASIP J. Adv. Signal Process..

[27]  Qingfu Zhang,et al.  A surrogate model assisted evolutionary algorithm for computationally expensive design optimization problems with discrete variables , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[28]  Bernhard Sendhoff,et al.  Generalizing Surrogate-Assisted Evolutionary Computation , 2010, IEEE Transactions on Evolutionary Computation.

[29]  Bernhard Sendhoff,et al.  A framework for evolutionary optimization with approximate fitness functions , 2002, IEEE Trans. Evol. Comput..

[30]  Jose E. Rayas-Sanchez,et al.  Polynomial-based surrogate modeling of microwave structures in frequency domain exploiting the multinomial theorem , 2016, 2016 IEEE MTT-S International Microwave Symposium (IMS).

[31]  Xin-She Yang,et al.  A literature survey of benchmark functions for global optimisation problems , 2013, Int. J. Math. Model. Numer. Optimisation.

[32]  Guoyuan Wu,et al.  Development and Evaluation of an Evolutionary Algorithm-Based OnLine Energy Management System for Plug-In Hybrid Electric Vehicles , 2017, IEEE Transactions on Intelligent Transportation Systems.

[33]  Xiaoyan Sun,et al.  Directed fuzzy graph-based surrogate model-assisted interactive genetic algorithms with uncertain individual's fitness , 2009, 2009 IEEE Congress on Evolutionary Computation.

[34]  Maxim A. Dulebenets,et al.  Application of Evolutionary Computation for Berth Scheduling at Marine Container Terminals: Parameter Tuning Versus Parameter Control , 2018, IEEE Transactions on Intelligent Transportation Systems.

[35]  Yang Yu,et al.  A two-layer surrogate-assisted particle swarm optimization algorithm , 2014, Soft Computing.

[36]  John Doherty,et al.  Committee-Based Active Learning for Surrogate-Assisted Particle Swarm Optimization of Expensive Problems , 2017, IEEE Transactions on Cybernetics.

[37]  Gianni Di Pillo,et al.  A SVM Surrogate Model-Based Method for Parametric Yield Optimization , 2016, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[38]  Michael N. Vrahatis,et al.  Recent approaches to global optimization problems through Particle Swarm Optimization , 2002, Natural Computing.

[39]  Geoffrey E. Hinton,et al.  Restricted Boltzmann machines for collaborative filtering , 2007, ICML '07.

[40]  Peter Tiño,et al.  Scaling Up Estimation of Distribution Algorithms for Continuous Optimization , 2011, IEEE Transactions on Evolutionary Computation.