Pymoo: Multi-Objective Optimization in Python

Python has become the programming language of choice for research and industry projects related to data science, machine learning, and deep learning. Since optimization is an inherent part of these research fields, more optimization related frameworks have arisen in the past few years. Only a few of them support optimization of multiple conflicting objectives at a time, but do not provide comprehensive tools for a complete multi-objective optimization task. To address this issue, we have developed pymoo, a multi-objective optimization framework in Python. We provide a guide to getting started with our framework by demonstrating the implementation of an exemplary constrained multi-objective optimization scenario. Moreover, we give a high-level overview of the architecture of pymoo to show its capabilities followed by an explanation of each module and its corresponding sub-modules. The implementations in our framework are customizable and algorithms can be modified/extended by supplying custom operators. Moreover, a variety of single, multi- and many-objective test problems are provided and gradients can be retrieved by automatic differentiation out of the box. Also, pymoo addresses practical needs, such as the parallelization of function evaluations, methods to visualize low and high-dimensional spaces, and tools for multi-criteria decision making. For more information about pymoo, readers are encouraged to visit: https://pymoo.org.

[1]  Georges G. Grinstein,et al.  Dimensional anchors: a graphic primitive for multidimensional multivariate information visualizations , 1999, NPIVM '99.

[2]  A. Wierzbicki.  A Mathematical Basis for Satisficing Decision Making , 1982 .

[3]  Richard J. Beckman,et al.  A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output From a Computer Code , 2000, Technometrics.

[4]  Ye Tian,et al.  PlatEMO: A MATLAB Platform for Evolutionary Multi-Objective Optimization [Educational Forum] , 2017, IEEE Computational Intelligence Magazine.

[5]  José Francisco Aldana Montes,et al.  jMetalCpp: optimizing molecular docking problems with a C++ metaheuristic framework , 2014, Bioinform..

[6]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[7]  Marc Parizeau,et al.  DEAP: evolutionary algorithms made easy , 2012, J. Mach. Learn. Res..

[8]  Sriparna Saha,et al.  Fast implementation of steady-state NSGA-II , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[9]  Eser Kandogan.  Star Coordinates: A Multi-dimensional Visualization Technique with Uniform Treatment of Dimensions , 2000 .

[10]  G. vanRossum.  Python reference manual , 1995 .

[11]  Antonio J. Nebro,et al.  jMetal: A Java framework for multi-objective optimization , 2011, Adv. Eng. Softw..

[12]  Kalyanmoy Deb,et al.  An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints , 2014, IEEE Transactions on Evolutionary Computation.

[13]  John E. Dennis,et al.  Normal-Boundary Intersection: A New Method for Generating the Pareto Surface in Nonlinear Multicriteria Optimization Problems , 1998, SIAM J. Optim..

[14]  Stefan Behnel,et al.  Cython: The Best of Both Worlds , 2011, Computing in Science & Engineering.

[15]  Dario Izzo,et al.  A Global Optimisation Toolbox for Massively Parallel Engineering Optimisation , 2010, ArXiv.

[16]  K. Deb,et al.  A bi-objective constrained optimization algorithm using a hybrid evolutionary and penalty function approach , 2013 .

[17]  Kalyanmoy Deb,et al.  Generating Well-Spaced Points on a Unit Simplex for Evolutionary Many-Objective Optimization , 2020, IEEE Transactions on Evolutionary Computation.

[18]  Niall M. Fraser,et al.  The modified star graph and the petal diagram: two new visual aids for discrete alternative multicriteria decision making , 1998 .

[19]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[20]  Joshua D. Knowles,et al.  On metrics for comparing nondominated sets , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[21]  Kalyanmoy Deb,et al.  Investigating the Normalization Procedure of NSGA-III , 2019, EMO.

[22]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[23]  Kalyanmoy Deb,et al.  A Unified Evolutionary Optimization Procedure for Single, Multiple, and Many Objectives , 2016, IEEE Transactions on Evolutionary Computation.

[24]  Kalyanmoy Deb,et al.  A flexible optimization procedure for mechanical component design based on genetic adaptive search , 1998 .

[25]  Ashwin Pajankar.  Python Unit Test Automation: Practical Techniques for Python Developers and Testers , 2017 .

[26]  Kalyanmoy Deb,et al.  Analysing mutation schemes for real-parameter genetic algorithms , 2014, Int. J. Artif. Intell. Soft Comput..

[27]  Lily Rachmawati,et al.  Multiobjective Evolutionary Algorithm With Controllable Focus on the Knees of the Pareto Front , 2009, IEEE Transactions on Evolutionary Computation.

[28]  Gary B. Lamont,et al.  Multiobjective evolutionary algorithms: classifications, analyses, and new innovations , 1999 .

[29]  James C. Bean,et al.  Genetic Algorithms and Random Keys for Sequencing and Optimization , 1994, INFORMS J. Comput..

[30]  Qingfu Zhang,et al.  MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition , 2007, IEEE Transactions on Evolutionary Computation.

[31]  E. Wegman.  Hyperdimensional Data Analysis Using Parallel Coordinates , 1990 .

[32]  Kalyanmoy Deb,et al.  Self-adaptive simulated binary crossover for real-parameter optimization , 2007, GECCO '07.

[33]  Guy L. Steele,et al.  The Java Language Specification, Java SE 8 Edition , 2013 .

[34]  Andrzej P. Wierzbicki,et al.  The Use of Reference Objectives in Multiobjective Optimization , 1979 .

[35]  Kalyanmoy Deb,et al.  A Running Performance Metric and Termination Criterion for Evaluating Evolutionary Multi- and Many-objective Optimization Algorithms , 2020, 2020 IEEE Congress on Evolutionary Computation (CEC).

[36]  Kalyanmoy Deb,et al.  Optimization for Engineering Design: Algorithms and Examples , 2004 .

[37]  Kalyanmoy Deb,et al.  An Optimality Theory-Based Proximity Measure for Set-Based Multiobjective Optimization , 2016, IEEE Transactions on Evolutionary Computation.

[38]  Hisao Ishibuchi,et al.  Modified Distance Calculation in Generational Distance and Inverted Generational Distance , 2015, EMO.

[39]  Lothar Thiele,et al.  The Hypervolume Indicator Revisited: On the Design of Pareto-compliant Indicators Via Weighted Integration , 2007, EMO.

[40]  R. K. Ursem.  Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[41]  Kalyanmoy Deb,et al.  Reference Point Based NSGA-III for Preferred Solutions , 2018, 2018 IEEE Symposium Series on Computational Intelligence (SSCI).

[42]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[43]  Lothar Thiele,et al.  Multiobjective Optimization Using Evolutionary Algorithms - A Comparative Case Study , 1998, PPSN.

[44]  Kalyanmoy Deb,et al.  Reference point based multi-objective optimization using evolutionary algorithms , 2006, GECCO.

[45]  John D. Hunter,et al.  Matplotlib: A 2D Graphics Environment , 2007, Computing in Science & Engineering.

[46]  Kalyanmoy Deb,et al.  An Interactive Evolutionary Multiobjective Optimization Method Based on Progressively Approximated Value Functions , 2010, IEEE Transactions on Evolutionary Computation.

[47]  Javier Del Ser,et al.  jMetalPy: a Python Framework for Multi-Objective Optimization with Metaheuristics , 2019, Swarm Evol. Comput..

[48]  Richard Curran,et al.  Optimization of noise abatement aircraft terminal routes using a multi-objective evolutionary algorithm based on decomposition , 2018 .

[49]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Comparing Review , 2006, Towards a New Evolutionary Computation.

[50]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[51]  Guy L. Steele,et al.  The Java Language Specification , 1996 .

[52]  Sanaz Mostaghim,et al.  Heatmap Visualization of Population Based Multi Objective Algorithms , 2007, EMO.

[53]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[54]  Kalyanmoy Deb,et al.  PaletteViz: A Visualization Method for Functional Understanding of High-Dimensional Pareto-Optimal Data-Sets to Aid Multi-Criteria Decision Making , 2020, IEEE Computational Intelligence Magazine.

[55]  Kalyanmoy Deb,et al.  A population-based fast algorithm for a billion-dimensional resource allocation problem with integer variables , 2017, Eur. J. Oper. Res..

[56]  D. Corne,et al.  On Metrics for Comparing Non Dominated Sets , 2001 .

[57]  Milan Zeleny,et al.  Gestalt system of holistic graphics: New management support view of MCDM , 1991, Comput. Oper. Res..

[58]  Bernabé Dorronsoro,et al.  A Survey of Decomposition Methods for Multi-objective Optimization , 2014, Recent Advances on Hybrid Approaches for Designing Intelligent Systems.

[59]  Kalyanmoy Deb,et al.  An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point Based Nondominated Sorting Approach, Part II: Handling Constraints and Extending to an Adaptive Approach , 2014, IEEE Transactions on Evolutionary Computation.

[60]  Uwe Naumann,et al.  Automatic Differentiation: Applications, Theory, and Implementations (Lecture Notes in Computational Science and Engineering) , 2006 .

[61]  Dario Izzo,et al.  FOR MASSIVELY PARALLEL OPTIMIZATION IN ASTRODYNAMICS ( THE CASE OF INTERPLANETARY TRAJECTORY OPTIMIZATION ) , 2012 .

[62]  Beat Kleiner,et al.  10 Graphical techniques for multivariate data and for clustering , 1982, Classification, Pattern Recognition and Reduction of Dimensionality.