A Random Forest-Assisted Evolutionary Algorithm for Data-Driven Constrained Multiobjective Combinatorial Optimization of Trauma Systems

Many real-world optimization problems can be solved by using the data-driven approach only, simply because no analytic objective functions are available for evaluating candidate solutions. In this paper, we address a class of expensive data-driven constrained multiobjective combinatorial optimization problems, where the objectives and constraints can be calculated only on the basis of a large amount of data. To solve this class of problems, we propose using random forests (RFs) and radial basis function networks as surrogates to approximate both objective and constraint functions. In addition, logistic regression models are introduced to rectify the surrogate-assisted fitness evaluations and a stochastic ranking selection is adopted to further reduce the influences of the approximated constraint functions. Three variants of the proposed algorithm are empirically evaluated on multiobjective knapsack benchmark problems and two real-world trauma system design problems. Experimental results demonstrate that the variant using RF models as the surrogates is effective and efficient in solving data-driven constrained multiobjective combinatorial optimization problems.

[1]  Yaochu Jin,et al.  A comprehensive survey of fitness approximation in evolutionary computation , 2005, Soft Comput..

[2]  John Doherty,et al.  Committee-Based Active Learning for Surrogate-Assisted Particle Swarm Optimization of Expensive Problems , 2017, IEEE Transactions on Cybernetics.

[3]  Kazuhiro Izui,et al.  Handling Undefined Vectors in Expensive Optimization Problems , 2010, EvoApplications.

[4]  Ahmed Kattan,et al.  Geometric Generalisation of Surrogate Model Based Optimisation to Combinatorial Spaces , 2011, EvoCOP.

[5]  Antanas Verikas,et al.  Mining data with random forests: A survey and results of new tests , 2011, Pattern Recognit..

[6]  Andy Liaw,et al.  Classification and Regression by randomForest , 2007 .

[7]  Roman Neruda,et al.  Feature Extraction for Surrogate Models in Genetic Programming , 2016, PPSN.

[8]  Oliver Kramer,et al.  Local SVM Constraint Surrogate Models for Self-adaptive Evolution Strategies , 2013, KI.

[9]  Andy J. Keane,et al.  Weld sequence optimization: The use of surrogate models for solving sequential combinatorial problems , 2005 .

[10]  Qingfu Zhang,et al.  A Gaussian Process Surrogate Model Assisted Evolutionary Algorithm for Medium Scale Expensive Optimization Problems , 2014, IEEE Transactions on Evolutionary Computation.

[11]  Jacques Teghem,et al.  The multiobjective multidimensional knapsack problem: a survey and a new approach , 2010, Int. Trans. Oper. Res..

[12]  Zhengdong Lu,et al.  Fast neural network surrogates for very high dimensional physics-based models in computational oceanography , 2007, Neural Networks.

[13]  Vipin Kumar,et al.  Similarity Measures for Categorical Data: A Comparative Evaluation , 2008, SDM.

[14]  Yaochu Jin,et al.  Metamodel Assisted Mixed-Integer Evolution Strategies Based on Kendall Rank Correlation Coefficient , 2013, IDEAL.

[15]  Patrick M. Reed,et al.  Diagnostic Assessment of Search Controls and Failure Modes in Many-Objective Evolutionary Optimization , 2012, Evolutionary Computation.

[16]  Andy J. Keane,et al.  Combining Global and Local Surrogate Models to Accelerate Evolutionary Optimization , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[17]  Bernhard Sendhoff,et al.  A study on metamodeling techniques, ensembles, and multi-surrogates in evolutionary computation , 2007, GECCO '07.

[18]  Rommel G. Regis,et al.  Evolutionary Programming for High-Dimensional Constrained Expensive Black-Box Optimization Using Radial Basis Functions , 2014, IEEE Transactions on Evolutionary Computation.

[19]  Bin Li,et al.  A New Memetic Algorithm With Fitness Approximation for the Defect-Tolerant Logic Mapping in Crossbar-Based Nanoarchitectures , 2014, IEEE Transactions on Evolutionary Computation.

[20]  Yaochu Jin,et al.  Incremental approximation of nonlinear constraint functions for evolutionary constrained optimization , 2010, IEEE Congress on Evolutionary Computation.

[21]  Marion K Campbell,et al.  The GEOS study: designing a geospatially optimised trauma system for Scotland. , 2014, The surgeon : journal of the Royal Colleges of Surgeons of Edinburgh and Ireland.

[22]  Handing Wang,et al.  Data-Driven Surrogate-Assisted Multiobjective Evolutionary Optimization of a Trauma System , 2016, IEEE Transactions on Evolutionary Computation.

[23]  Karla L. Homan Combinatorial optimization: Current successes and directions for the future , 2000 .

[24]  Serpil Sayin,et al.  A new algorithm for generating all nondominated solutions of multiobjective discrete optimization problems , 2014, Eur. J. Oper. Res..

[25]  George Mavrotas,et al.  Solving multiobjective, multiconstraint knapsack problems using mathematical programming and evolutionary algorithms , 2010, Eur. J. Oper. Res..

[26]  James M. Parr,et al.  Infill sampling criteria for surrogate-based optimization with constraint handling , 2012 .

[27]  Kwang-Yong Kim,et al.  Enhanced multi-objective optimization of a microchannel heat sink through evolutionary algorithm coupled with multiple surrogate models , 2010 .

[28]  K. Gupta,et al.  Metamodel-Based Optimization for Problems With Expensive Objective and Constraint Functions , 2011 .

[29]  Yaochu Jin,et al.  Surrogate-Assisted Multicriteria Optimization: Complexities, Prospective Solutions, and Business Case , 2017 .

[30]  Yew-Soon Ong,et al.  A study on polynomial regression and Gaussian process global surrogate model in hierarchical surrogate-assisted evolutionary algorithm , 2005, 2005 IEEE Congress on Evolutionary Computation.

[31]  Nenad Mladenovic,et al.  Multi-objective variable neighborhood search: an application to combinatorial optimization problems , 2015, J. Glob. Optim..

[32]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[33]  Slawomir Koziel,et al.  Rapid Simulation-Driven Multiobjective Design Optimization of Decomposable Compact Microwave Passives , 2016, IEEE Transactions on Microwave Theory and Techniques.

[34]  Xavier Gandibleux,et al.  A survey and annotated bibliography of multiobjective combinatorial optimization , 2000, OR Spectr..

[35]  Ye Tian,et al.  Effectiveness and efficiency of non-dominated sorting for evolutionary multi- and many-objective optimization , 2017, Complex & Intelligent Systems.

[36]  Bernhard Sendhoff,et al.  On Evolutionary Optimization with Approximate Fitness Functions , 2000, GECCO.

[37]  Daniel Vanderpooten,et al.  Solving efficiently the 0-1 multi-objective knapsack problem , 2009, Comput. Oper. Res..

[38]  Yew-Soon Ong,et al.  A surrogate-assisted memetic co-evolutionary algorithm for expensive constrained optimization problems , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[39]  Mengjie Zhang,et al.  Surrogate-Assisted Genetic Programming With Simplified Models for Automated Design of Dispatching Rules , 2017, IEEE Transactions on Cybernetics.

[40]  Bernhard Sendhoff,et al.  Generalizing Surrogate-Assisted Evolutionary Computation , 2010, IEEE Transactions on Evolutionary Computation.

[41]  David A. Freedman,et al.  Statistical Models: Theory and Practice: References , 2005 .

[42]  Chee Keong Kwoh,et al.  Feasibility Structure Modeling: An Effective Chaperone for Constrained Memetic Algorithms , 2010, IEEE Transactions on Evolutionary Computation.

[43]  Bernhard Sendhoff,et al.  A framework for evolutionary optimization with approximate fitness functions , 2002, IEEE Trans. Evol. Comput..

[44]  Bernhard Sendhoff,et al.  Structure optimization of neural networks for evolutionary design optimization , 2005, Soft Comput..

[45]  Qingfu Zhang,et al.  Expensive Multiobjective Optimization by MOEA/D With Gaussian Process Model , 2010, IEEE Transactions on Evolutionary Computation.

[46]  Qingfu Zhang,et al.  MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition , 2007, IEEE Transactions on Evolutionary Computation.

[47]  Yang Yu,et al.  A two-layer surrogate-assisted particle swarm optimization algorithm , 2014, Soft Computing.

[48]  Yaochu Jin,et al.  Surrogate-assisted evolutionary computation: Recent advances and future challenges , 2011, Swarm Evol. Comput..

[49]  Jianchao Zeng,et al.  Surrogate-Assisted Cooperative Swarm Optimization of High-Dimensional Expensive Problems , 2017, IEEE Transactions on Evolutionary Computation.

[50]  Jürgen Branke,et al.  Faster convergence by means of fitness estimation , 2005, Soft Comput..

[51]  Carlos Artemio Coello-Coello,et al.  Theoretical and numerical constraint-handling techniques used with evolutionary algorithms: a survey of the state of the art , 2002 .

[52]  Oliver Kramer,et al.  Surrogate Constraint Functions for CMA Evolution Strategies , 2009, KI.

[53]  Joshua D. Knowles,et al.  ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems , 2006, IEEE Transactions on Evolutionary Computation.

[54]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[55]  Andy J. Keane,et al.  Enhancing infill sampling criteria for surrogate-based constrained optimization , 2012, J. Comput. Methods Sci. Eng..

[56]  Ye Tian,et al.  A Classification-Based Surrogate-Assisted Evolutionary Algorithm for Expensive Many-Objective Optimization , 2019, IEEE Transactions on Evolutionary Computation.

[57]  D. Steinberg CART: Classification and Regression Trees , 2009 .

[58]  A. Keane,et al.  Evolutionary Optimization of Computationally Expensive Problems via Surrogate Modeling , 2003 .

[59]  D. Wolfe,et al.  Nonparametric Statistical Methods. , 1974 .

[60]  Feng Liu,et al.  A knowledge-based evolutionary proactive scheduling approach in the presence of machine breakdown and deterioration effect , 2015, Knowl. Based Syst..

[61]  Kaisa Miettinen,et al.  A Surrogate-Assisted Reference Vector Guided Evolutionary Algorithm for Computationally Expensive Many-Objective Optimization , 2018, IEEE Transactions on Evolutionary Computation.

[62]  John Doherty,et al.  Hierarchical Surrogate-Assisted Evolutionary Multi-Scenario Airfoil Shape Optimization , 2018, 2018 IEEE Congress on Evolutionary Computation (CEC).

[63]  Kaisa Miettinen,et al.  On Constraint Handling in Surrogate-Assisted Evolutionary Many-Objective Optimization , 2016, PPSN.

[64]  Qingfu Zhang,et al.  Multiobjective optimization Test Instances for the CEC 2009 Special Session and Competition , 2009 .

[65]  Xin Yao,et al.  Corner Sort for Pareto-Based Many-Objective Optimization , 2014, IEEE Transactions on Cybernetics.

[66]  Michael T. M. Emmerich,et al.  Mixed-integer optimization of coronary vessel image analysis using evolution strategies , 2006, GECCO '06.

[67]  John Doherty,et al.  Offline Data-Driven Evolutionary Optimization Using Selective Surrogate Ensembles , 2019, IEEE Transactions on Evolutionary Computation.

[68]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[69]  Eckart Zitzler,et al.  HypE: An Algorithm for Fast Hypervolume-Based Many-Objective Optimization , 2011, Evolutionary Computation.

[70]  Xin Yao,et al.  Stochastic ranking for constrained evolutionary optimization , 2000, IEEE Trans. Evol. Comput..

[71]  Thomas Bartz-Beielstein,et al.  Model-based methods for continuous and discrete global optimization , 2017, Appl. Soft Comput..

[72]  Kevin Leyton-Brown,et al.  Sequential Model-Based Optimization for General Algorithm Configuration , 2011, LION.

[73]  Tapabrata Ray,et al.  Surrogate assisted Simulated Annealing (SASA) for constrained multi-objective optimization , 2010, IEEE Congress on Evolutionary Computation.

[74]  E. L. Ulungu,et al.  Multi‐objective combinatorial optimization problems: A survey , 1994 .

[75]  Kaisa Miettinen,et al.  Nonlinear multiobjective optimization , 1998, International series in operations research and management science.

[76]  Qingfu Zhang,et al.  Multiobjective evolutionary algorithms: A survey of the state of the art , 2011, Swarm Evol. Comput..

[77]  T. Ray,et al.  A framework for design optimization using surrogates , 2005 .

[78]  Bernhard Sendhoff,et al.  Comparing neural networks and Kriging for fitness approximation in evolutionary optimization , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[79]  Yong Wang,et al.  A Multiobjective Optimization-Based Evolutionary Algorithm for Constrained Optimization , 2006, IEEE Transactions on Evolutionary Computation.

[80]  Michael T. M. Emmerich,et al.  Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels , 2006, IEEE Transactions on Evolutionary Computation.

[81]  Dan Guo,et al.  Small data driven evolutionary multi-objective optimization of fused magnesium furnaces , 2016, 2016 IEEE Symposium Series on Computational Intelligence (SSCI).

[82]  Bernhard Sendhoff,et al.  A systems approach to evolutionary multiobjective structural optimization and beyond , 2009, IEEE Computational Intelligence Magazine.