Metamodeling by symbolic regression and Pareto simulated annealing

The subject of this paper is a new approach to symbolic regression. Other publications on symbolic regression use genetic programming. This paper describes an alternative method based on Pareto simulated annealing. Our method is based on linear regression for the estimation of constants. Interval arithmetic is applied to ensure the consistency of a model. To prevent overfitting, we merit a model not only on predictions in the data points, but also on the complexity of a model. For the complexity, we introduce a new measure. We compare our new method with the Kriging metamodel and against a symbolic regression metamodel based on genetic programming. We conclude that Pareto-simulated-annealing-based symbolic regression is very competitive compared to the other metamodel approaches.

[1]  Annie A. M. Cuyt,et al.  Multivariate rational data fitting: general data structure, maximal accuracy and object orientation , 1992, Numerical Algorithms.

[2]  T. Simpson,et al.  Comparative studies of metamodeling techniques under multiple modeling criteria , 2000 .

[3]  Margaret J. Robertson,et al.  Design and Analysis of Experiments , 2006, Handbook of statistics.

[4]  Piotr Czyzżak,et al.  Pareto simulated annealing—a metaheuristic technique for multiple‐objective combinatorial optimization , 1998 .

[5]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[6]  Jonathan E. Fieldsend,et al.  Dominance measures for multi-objective simulated annealing , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[7]  Xiaoping Du,et al.  The use of metamodeling techniques for optimization under uncertainty , 2001 .

[8]  T. Simpson,et al.  Comparative studies of metamodelling techniques under multiple modelling criteria , 2001 .

[9]  Thomas J. Santner,et al.  The Design and Analysis of Computer Experiments , 2003, Springer Series in Statistics.

[10]  Maarten Keijzer,et al.  Improving Symbolic Regression with Interval Arithmetic and Linear Scaling , 2003, EuroGP.

[11]  J. -F. M. Barthelemy,et al.  Approximation concepts for optimum structural design — a review , 1993 .

[12]  T. J. Mitchell,et al.  Exploratory designs for computational experiments , 1995 .

[13]  C. McDiarmid.  SIMULATED ANNEALING AND BOLTZMANN MACHINES A Stochastic Approach to Combinatorial Optimization and Neural Computing , 1991 .

[14]  Mark Kotanchek,et al.  Pareto-Front Exploitation in Symbolic Regression , 2005 .

[15]  Thomas J. Santner,et al.  Design and analysis of computer experiments , 1998 .

[16]  Ren-Jye Yang,et al.  Approximation methods in multidisciplinary analysis and optimization: a panel discussion , 2004 .

[17]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[18]  M. J. D. Powell,et al.  Radial basis functions for multivariable interpolation: a review , 1987 .

[19]  Alexander J. Smola,et al.  Support Vector Method for Function Approximation, Regression Estimation and Signal Processing , 1996, NIPS.

[20]  N. M. Alexandrov,et al.  A trust-region framework for managing the use of approximation models in optimization , 1997 .

[21]  George Lindfield,et al.  Numerical Methods Using MATLAB , 1998 .