Variational Inference with Vine Copulas: An efficient Approach for Bayesian Computer Model Calibration

With the advancements of computer architectures, the use of computational models proliferates to solve complex problems in many scientific applications such as nuclear physics and climate research. However, the potential of such models is often hindered because they tend to be computationally expensive and consequently ill-fitting for uncertainty quantification. Furthermore, they are usually not calibrated with real-time observations. We develop a computationally efficient algorithm based on variational Bayes inference (VBI) for calibration of computer models with Gaussian processes. Unfortunately, the speed and scalability of VBI diminishes when applied to the calibration framework with dependent data. To preserve the efficiency of VBI, we adopt a pairwise decomposition of the data likelihood using vine copulas that separate the information on dependence structure in data from their marginal distributions. We provide both theoretical and empirical evidence for the computational scalability of our methodology and describe all the necessary details for an efficient implementation of the proposed algorithm. We also demonstrated the opportunities given by our method for practitioners on a real data example through calibration of the Liquid Drop Model of nuclear binding energies.

[1]  A. O'Hagan,et al.  Bayesian calibration of computer models , 2001 .

[2]  T. Bedford,et al.  Vines: A new graphical model for dependent random variables , 2002 .

[3]  David M. Blei,et al.  Frequentist Consistency of Variational Bayes , 2017, Journal of the American Statistical Association.

[4]  T. J. Mitchell,et al.  Exploratory designs for computational experiments , 1995 .

[5]  L. Neufcourt,et al.  Direct Comparison between Bayesian and Frequentist Uncertainty Quantification for Nuclear Reactions. , 2019, Physical review letters.

[6]  S. Chib,et al.  Understanding the Metropolis-Hastings Algorithm , 1995 .

[7]  Tianqi Chen,et al.  A Complete Recipe for Stochastic Gradient MCMC , 2015, NIPS.

[8]  Carsten Peterson,et al.  A Mean Field Theory Learning Algorithm for Neural Networks , 1987, Complex Syst..

[9]  Andrew Gelman,et al.  The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo , 2011, J. Mach. Learn. Res..

[10]  D. Bingham,et al.  Estimating Parameter Uncertainty in Binding-Energy Models by the Frequency-Domain Bootstrap. , 2017, Physical review letters.

[11]  Emily B. Fox,et al.  Stochastic Gradient MCMC for State Space Models , 2018, SIAM J. Math. Data Sci..

[12]  H. Robbins A Stochastic Approximation Method , 1951 .

[13]  M. Bender,et al.  From finite nuclei to the nuclear liquid drop: Leptodermous expansion based on self-consistent mean-field theory , 2006 .

[14]  Giuseppe Ragusa,et al.  Econometric Theory , 2013 .

[15]  W. D. Myers,et al.  NUCLEAR MASSES AND DEFORMATIONS , 1966 .

[16]  Edoardo M. Airoldi,et al.  Copula variational inference , 2015, NIPS.

[17]  Stefan M. Wild,et al.  Nuclear energy density optimization: Shell structure , 2013, 1312.1746.

[18]  Earl Lawrence,et al.  THE COYOTE UNIVERSE. III. SIMULATION SUITE AND PRECISION EMULATOR FOR THE NONLINEAR MATTER POWER SPECTRUM , 2009, 0912.4490.

[19]  Dave Higdon,et al.  Combining Field Data and Computer Simulations for Calibration and Prediction , 2005, SIAM J. Sci. Comput..

[20]  Michalis K. Titsias,et al.  Variational Learning of Inducing Variables in Sparse Gaussian Processes , 2009, AISTATS.

[21]  Vojtech Kejzlar,et al.  Statistical aspects of nuclear mass models , 2020 .

[22]  Stefan M. Wild,et al.  Uncertainty quantification for nuclear density functional theory and information content of new measurements. , 2015, Physical review letters.

[23]  G. Casella,et al.  Rao-Blackwellisation of sampling schemes , 1996 .

[24]  Hui Yang,et al.  Calibrating Functional Parameters in the Ion Channel Models of Cardiac Cells , 2016 .

[25]  S. Bentridi,et al.  Bethe–Weizsäcker semiempirical mass formula coefficients 2019 update based on AME2016 , 2020 .

[26]  Jongeun Choi,et al.  Patient-Specific Prediction of Abdominal Aortic Aneurysm Expansion Using Bayesian Calibration , 2019, IEEE Journal of Biomedical and Health Informatics.

[27]  David M. Blei,et al.  Stochastic Structured Variational Inference , 2014, AISTATS.

[28]  A. H. Wapstra,et al.  The AME2003 atomic mass evaluation . (II). Tables, graphs and references , 2003 .

[29]  M. Plumlee Computer model calibration with confidence and consistency , 2019, Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[30]  Hans A. Bethe,et al.  Nuclear Physics A. Stationary States of Nuclei , 1936 .

[31]  James O. Berger,et al.  A Framework for Validation of Computer Models , 2007, Technometrics.

[32]  C. Czado,et al.  Truncated regular vines in high dimensions with application to financial data , 2012 .

[33]  Wei Deng,et al.  An Adaptive Empirical Bayesian Method for Sparse Deep Learning , 2019, NeurIPS.

[34]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine Learning.

[35]  Long Wang,et al.  Scaled Gaussian Stochastic Process for Computer Model Calibration and Prediction , 2017, SIAM/ASA J. Uncertain. Quantification.

[36]  Claudia Czado,et al.  Selecting and estimating regular vine copulae and application to financial returns , 2012, Comput. Stat. Data Anal..

[37]  Stefan M. Wild,et al.  A Bayesian approach for parameter estimation and prediction using a computationally intensive model , 2014, Journal of Physics G: Nuclear and Particle Physics.

[38]  David J. Nott,et al.  High-Dimensional Copula Variational Approximation Through Transformation , 2019 .

[39]  Yoshua Bengio,et al.  Global training of document processing systems using graph transformer networks , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[40]  G.F.Bertsch,et al.  Fitting theories of nuclear binding energies , 2004, nucl-th/0412091.

[41]  Collin Carbno,et al.  Uncertainty Analysis With High Dimensional Dependence Modelling , 2007, Technometrics.

[42]  M. Plumlee Bayesian Calibration of Inexact Computer Models , 2017 .

[43]  Dustin Tran,et al.  Hierarchical Variational Models , 2015, ICML.

[44]  Chong Wang,et al.  Asymptotically Exact, Embarrassingly Parallel MCMC , 2013, UAI.

[45]  M. Webb,et al.  Multivariate probabilistic projections using imperfect climate models part I: outline of methodology , 2012, Climate Dynamics.

[46]  Murali Haran,et al.  Large ensemble modeling of the last deglacial retreat of the West Antarctic Ice Sheet: comparison of simple and advanced statistical techniques , 2015 .

[47]  D. Higdon,et al.  Computer Model Calibration Using High-Dimensional Output , 2008 .

[48]  E. Gadioli,et al.  Introductory Nuclear Physics , 1997 .

[49]  M. Kirson Mutual influence of terms in a semi-empirical mass formula , 2008 .

[50]  Sean Gerrish,et al.  Black Box Variational Inference , 2013, AISTATS.

[51]  Chong Wang,et al.  Stochastic variational inference , 2012, J. Mach. Learn. Res..

[52]  Stefan M. Wild,et al.  Nuclear energy density optimization: Large deformations , 2011, 1111.4344.

[53]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[54]  S. Fayans Towards a universal nuclear density functional , 1998 .

[55]  J. Barkoulas,et al.  Econometric Methods I , 2015 .

[56]  Carl E. Rasmussen,et al.  A Unifying View of Sparse Approximate Gaussian Process Regression , 2005, J. Mach. Learn. Res..

[57]  C. Yuan A statistical view on nuclear mass formula based on liquid drop model , 2014, 1407.3970.

[58]  Harry Joe,et al.  Truncation of vine copulas using fit indices , 2015, J. Multivar. Anal..

[59]  Roger M. Cooke,et al.  Uncertainty Analysis with High Dimensional Dependence Modelling , 2006 .

[60]  M. Sklar Fonctions de repartition a n dimensions et leurs marges , 1959 .

[61]  David M. Blei,et al.  Overdispersed Black-Box Variational Inference , 2016, UAI.

[62]  Matthew D. Zeiler ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[63]  A. H. Wapstra,et al.  The Ame2003 atomic mass evaluation: (I). Evaluation of input data, adjustment procedures☆ , 2003 .

[64]  J. Dobaczewski,et al.  Error Estimates of Theoretical Models: a Guide , 2014, 1402.4657.

[65]  Kjersti Aas,et al.  Truncated regular vines in high dimensions , 2010 .

[66]  C. Weizsäcker Zur Theorie der Kernmassen , 1935 .

[67]  Stefan M. Wild,et al.  Nuclear Energy Density Optimization , 2010, 1005.5145.