An adjoint for likelihood maximization

The process of likelihood maximization can be found in many different areas of computational modelling. However, the construction of such models via likelihood maximization requires the solution of a difficult multi-modal optimization problem involving an expensive O(n3) factorization. The optimization techniques used to solve this problem may require many such factorizations and can result in a significant bottleneck. This article derives an adjoint formulation of the likelihood employed in the construction of a kriging model via reverse algorithmic differentiation. This adjoint is found to calculate the likelihood and all of its derivatives more efficiently than the standard analytical method and can therefore be used within a simple local search or within a hybrid global optimization to accelerate convergence and therefore reduce the cost of the likelihood optimization.

[1]  D. Krige.  A statistical approach to some basic mine valuation problems on the Witwatersrand, by D.G. Krige, published in the Journal, December 1951 : introduction by the author , 1951 .

[2]  C. G. Broyden.  The Convergence of a Class of Double-rank Minimization Algorithms 1. General Considerations , 1970 .

[3]  C. G. Broyden.  The Convergence of a Class of Double-rank Minimization Algorithms 2. The New Algorithm , 1970 .

[4]  W. Press,et al.  Numerical Recipes: The Art of Scientific Computing , 1987 .

[5]  Koichi Kubota,et al.  Matrix inversion algorithms by means of automatic differentiation , 1994 .

[6]  S. P. Smith.  Differentiation of the Cholesky Algorithm , 1995 .

[7]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[8]  Andreas Griewank,et al.  Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition , 2000, Frontiers in applied mathematics.

[9]  Timothy W. Simpson,et al.  Metamodels for Computer-based Engineering Design: Survey and recommendations , 2001, Engineering with Computers.

[10]  Jangsun Baek,et al.  Efficient computation of maximum likelihood estimators in a spatial linear model with power exponential covariogram , 2001 .

[11]  Donald R. Jones,et al.  A Taxonomy of Global Optimization Methods Based on Response Surfaces , 2001, J. Glob. Optim..

[12]  T. Simpson,et al.  Comparative studies of metamodelling techniques under multiple modelling criteria , 2001 .

[13]  Peter Hollingsworth,et al.  Gaussian Process Meta-modeling: Comparison of Gausian Process Training Methods , 2003 .

[14]  Thomas J. Santner,et al.  Design and analysis of computer experiments , 1998 .

[15]  M. Zako,et al.  Structural optimization using Kriging approximation , 2003 .

[16]  Thomas J. Santner,et al.  The Design and Analysis of Computer Experiments , 2003, Springer Series in Statistics.

[17]  Ranjan Ganguli,et al.  An automated hybrid genetic-conjugate gradient algorithm for multimodal optimization problems , 2005, Appl. Math. Comput..

[18]  T. Simpson,et al.  Use of Kriging Models to Approximate Deterministic Computer Models , 2005 .

[19]  Edmondo A. Minisci,et al.  Multi-objective evolutionary optimization of subsonic airfoils by kriging approximation and evolution control , 2005, 2005 IEEE Congress on Evolutionary Computation.

[20]  Raphael T. Haftka,et al.  Surrogate-based Analysis and Optimization , 2005 .

[21]  Yunong Zhang,et al.  Exploiting Hessian matrix and trust-region algorithm in hyperparameters estimation of Gaussian process , 2005, Appl. Math. Comput..

[22]  John E. Renaud,et al.  Update Strategies for Kriging Models for Use in Variable Fidelity Optimization , 2005 .

[23]  Andy J. Keane,et al.  Statistical Improvement Criteria for Use in Multiobjective Design Optimization , 2006 .

[24]  G. Gary Wang,et al.  Review of Metamodeling Techniques in Support of Engineering Design Optimization , 2007, DAC 2006.

[25]  Kaare Brandt Petersen,et al.  The Matrix Cookbook , 2006 .

[26]  Shawn E. Gano,et al.  Update strategies for kriging models used in variable fidelity optimization , 2006 .

[27]  J. Alonso,et al.  ADjoint: An Approach for the Rapid Development of Discrete Adjoint Solvers , 2006 .

[28]  R. A. Miller,et al.  Sequential kriging optimization using multiple-fidelity evaluations , 2006 .

[29]  于海斌,et al.  A hybrid PSO-GD based intelligent method for machine diagnosis , 2006 .

[30]  A. Forrester,et al.  Design and analysis of 'noisy' computer experiments , 2006 .

[31]  Andy J. Keane,et al.  Design Optimization of a Two-Dimensional Subsonic Engine Air Intake , 2006 .

[32]  Andy J. Keane,et al.  Optimization using surrogate models and partially converged computational fluid dynamics simulations , 2006, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[33]  Edmondo Minisci,et al.  Multi-objective evolutionary optimization of subsonic airfoils by meta-modelling and evolution control , 2006 .

[34]  Yunong Zhang,et al.  O(N 2)-Operation Approximation of Covariance Matrix Inverse in Gaussian Process Regression Based on Quasi-Newton BFGS Method , 2007, Commun. Stat. Simul. Comput..

[35]  Andy J. Keane,et al.  Engineering Design via Surrogate Modelling - A Practical Guide , 2008 .

[36]  Andy J. Keane,et al.  Geometric filtration using POD for aerodynamic design optimization , 2008 .

[37]  David J. J. Toal,et al.  Kriging Hyperparameter Tuning Strategies , 2008 .

[38]  Matemática,et al.  Society for Industrial and Applied Mathematics , 2010 .