An Information-Theoretic Approach to Optimally Calibrate Approximate Models

With the advancements in modeling and numerical algorithms, the decision supported by modeling and simulation has became more mainstream than ever. Even though the computational power is continually increased, in most engineering applications the problem of optimal design under uncertainty has became prohibitively expensive due to long runtimes of single simulations. The obvious solution is to reduce the complexity of the model by employing di erent assumptions and constructing this way an approximate model. The calibration of these simpler models requires a large number of runs of the complex model, which may still be too expensive and ine cient for the task at hand. In this paper, we study the problem of optimal data collection to e ciently learn the model parameters of an approximate model in the context of Bayesian analysis. The paper emphasizes the in uence of model discrepancy on the calibration of the approximate model and hence the choice of optimal designs. Model discrepancy is modeled using a Gaussian process in this study. The optimal design is obtained as a result of an information theoretic sensitivity analysis. Thus, the preferred design is where the statistical dependence between the model parameters and observables is the highest possible. In this paper, the statistical dependence between random variables is quanti ed by mutual information and estimated using a k-nearest neighbor based approximation. As a model problem, a convective-dispersion model is calibrated to approximate the physics of Burgers’ equation in a limited time domain of interest.