Model-agnostic multi-objective approach for the evolutionary discovery of mathematical models

In modern data science, it is often not enough to obtain only a data-driven model with a good prediction quality. On the contrary, it is more interesting to understand the properties of the model, which parts could be replaced to obtain better results. Such questions are unified under machine learning interpretability questions, which could be considered one of the area’s raising topics. In the paper, we use multi-objective evolutionary optimization for composite data-driven model learning to obtain the algorithm’s desired properties. It means that whereas one of the apparent objectives is precision, the other could be chosen as the complexity of the model, robustness, and many others. The method application is shown on examples of multi-objective learning of composite models, differential equations, and closed-form algebraic expressions are unified and form approach for model-agnostic learning of the interpretable models.

[1]  A. Hvatov,et al.  Closed-form algebraic expressions discovery using combined evolutionary optimization and sparse regression approach , 2020 .

[2]  Frank Hutter,et al.  Neural Architecture Search: A Survey , 2018, J. Mach. Learn. Res..

[3]  Tuong Manh Vu,et al.  Toward inverse generative social science using multi-objective genetic programming , 2019, GECCO.

[4]  D. Shahsavani,et al.  Variance-based sensitivity analysis of model outputs using surrogate models , 2011, Environ. Model. Softw..

[5]  Nikolay O. Nikitin,et al.  Adaptation of NEMO-LIM3 model for multigrid high resolution Arctic simulation , 2018, Ocean Modelling.

[6]  Jun Ren,et al.  Using Genetic Programming with Prior Formula Knowledge to Solve Symbolic Regression Problem , 2015, Comput. Intell. Neurosci..

[7]  Zachary Chase Lipton The mythos of model interpretability , 2016, ACM Queue.

[8]  Anna V. Kaluzhnaya,et al.  Partial differential equations discovery with EPDE framework: Application for real and synthetic data , 2021, J. Comput. Sci..

[9]  Evolving mathematical expressions using Genetic Algorithms , 2004 .

[10]  Randal S. Olson,et al.  TPOT: A Tree-based Pipeline Optimization Tool for Automating Machine Learning , 2016, AutoML@ICML.

[11]  Structural Evolutionary Learning for Composite Classification Models , 2020, Procedia Computer Science.

[12]  Nikolay O. Nikitin,et al.  Automatic evolutionary learning of composite models with knowledge enrichment , 2020, GECCO Companion.

[13]  Boaz Lerner,et al.  Inference Graphs for CNN Interpretation , 2021, ECCV.

[14]  Paola Annoni,et al.  Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index , 2010, Comput. Phys. Commun..

[15]  Qingfu Zhang,et al.  An Evolutionary Many-Objective Optimization Algorithm Based on Dominance and Decomposition , 2015, IEEE Transactions on Evolutionary Computation.

[16]  Nikolay O. Nikitin,et al.  REBEC: Robust Evolutionary-based Calibration Approach for the Numerical Wind Wave Model , 2019, ArXiv.

[17]  Stelios Kapetanakis,et al.  Artificial Neural Network and Multiple Linear Regression for Flood Prediction in Mohawk River, New York , 2018, Water.