Efficient Global Optimization with Indefinite Kernels

Kernel based surrogate models like Kriging are a popular remedy for costly objective function evaluations in optimization. Often, kernels are required to be definite. Highly customized kernels, or kernels for combinatorial representations, may be indefinite. This study investigates this issue in the context of Kriging. It is shown that approaches from the field of Support Vector Machines are useful starting points, but require further modifications to work with Kriging. This study compares a broad selection of methods for dealing with indefinite kernels in Kriging and Kriging-based Efficient Global Optimization, including spectrum transformation, feature embedding and computation of the nearest definite matrix. Model quality and optimization performance are tested. The standard, without explicitly correcting indefinite matrices, yields functional results, which are further improved by spectrum transformations.

[1]  Yaochu Jin,et al.  A comprehensive survey of fitness approximation in evolutionary computation , 2005, Soft Comput..

[2]  C. D. Perttunen,et al.  Lipschitzian optimization without the Lipschitz constant , 1993 .

[3]  Thomas Bartz-Beielstein,et al.  Efficient global optimization for combinatorial problems , 2014, GECCO.

[4]  Robert Tibshirani,et al.  1-norm Support Vector Machines , 2003, NIPS.

[5]  N. Higham.  Computing the nearest correlation matrix—a problem from finance , 2002 .

[6]  Andy J. Keane,et al.  Engineering Design via Surrogate Modelling - A Practical Guide , 2008 .

[7]  T. W. Anderson.  On the Distribution of the Two-Sample Cramer-von Mises Criterion , 1962 .

[8]  Claus Bahlmann,et al.  Learning with Distance Substitution Kernels , 2004, DAGM-Symposium.

[9]  Clayton V. Deutsch,et al.  Programs for kriging and sequential Gaussian simulation with locally varying anisotropy using non-Euclidean distances , 2011, Comput. Geosci..

[10]  Alberto Moraglio,et al.  Geometric surrogate-based optimisation for permutation-based problems , 2011, GECCO '11.

[11]  Rodolphe Le Riche,et al.  An analytic comparison of regularization methods for Gaussian Processes , 2016, 1602.00853.

[12]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[13]  C. Chu,et al.  Towards Indefinite Gaussian Processes , 2012 .

[14]  Brian S. Yandell,et al.  Practical Data Analysis for Designed Experiments , 1998 .

[15]  Peter Tiño,et al.  Indefinite Proximity Learning: A Review , 2015, Neural Computation.

[16]  Zheng Zhang,et al.  An Analysis of Transformation on Non - Positive Semidefinite Similarity Matrix for Kernel Machines , 2005, ICML 2005.

[17]  Cheng Soon Ong,et al.  Learning SVM in Kreĭn Spaces , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Olvi L. Mangasarian,et al.  Generalized Support Vector Machines , 1998 .

[19]  Bernhard Schölkopf,et al.  The Kernel Trick for Distances , 2000, NIPS.

[20]  Maya R. Gupta,et al.  Learning kernels from indefinite similarities , 2009, ICML '09.

[21]  W. Glunt,et al.  An alternating projection algorithm for computing the nearest euclidean distance matrix , 1990 .

[22]  Clayton V. Deutsch,et al.  Robust Solution of Normal ( Kriging ) Equations , 2007 .

[23]  R. Rebonato,et al.  The Most General Methodology to Create a Valid Correlation Matrix for Risk Management and Option Pricing Purposes , 2011 .

[24]  Ahmed Kattan,et al.  Geometric Generalisation of Surrogate Model Based Optimisation to Combinatorial Spaces , 2011, EvoCOP.

[25]  Thomas Bartz-Beielstein,et al.  Distance Measures for Permutations in Combinatorial Efficient Global Optimization , 2014, PPSN.

[26]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.