Computationally Inexpensive Metamodel Assessment Strategies

In many scientific and engineering domains, it is common to analyze and simulate complex physical systems using mathematical models. Although computing resources continue to increase in power and speed, computer simulation and analysis codes continue to grow in complexity and remain computationally expensive, limiting their use in design and optimization. Consequently, many researchers have developed different metamodeling strategies to create inexpensive approximations of computationally expensive computer simulations. These approximations introduce a new element of uncertainty during design optimization, and there is a need to develop efficient methods to assess metamodel validity. We investigate computationally inexpensive assessment methods for metamodel validation based on leave-k-out cross validation and develop guidelines for selecting k for different types of metamodels. Based on the results from two sets of test problems, k = 1 is recommended for leave-k-out cross validation of low-order polynomial and radial basis function metamodels, whereas k=0.1N or N is recommended for kriging metamodels, where N is the number of sample points used to construct the metamodel.