Attentive Neural Processes and Batch Bayesian Optimization for Scalable Calibration of Physics-Informed Digital Twins

Physics-informed dynamical system models form critical components of digital twins of the built environment. These digital twins enable the design of energy-efficient infrastructure, but must be properly calibrated to accurately reflect system behavior for downstream prediction and analysis. Dynamical system models of modern buildings are typically described by a large number of parameters and incur significant computational expenditure during simulations. To handle large-scale calibration of digital twins without exorbitant simulations, we propose ANP-BBO: a scalable and parallelizable batch-wise Bayesian optimization (BBO) methodology that leverages attentive neural processes (ANPs).

[1]  Yoshua Bengio,et al.  Tackling Climate Change with Machine Learning , 2019, ACM Comput. Surv..

[2]  Tuan Anh Le,et al.  Empirical Evaluation of Neural Process Objectives , 2018 .

[3]  Yuxiang Chen,et al.  Data-driven modeling of building thermal dynamics: Methodology and state of the art , 2019, Energy and Buildings.

[4]  Jasper Snoek,et al.  Practical Bayesian Optimization of Machine Learning Algorithms , 2012, NIPS.

[5]  Prabhat,et al.  Scalable Bayesian Optimization Using Deep Neural Networks , 2015, ICML.

[6]  Thierry S. Nouidui,et al.  Modelica Buildings library , 2014 .

[7]  Wencheng Wu,et al.  Neural Process for Black-box Model Optimization Under Bayesian Framework , 2021, AAAI Spring Symposium: MLPS.

[8]  Yee Whye Teh,et al.  Attentive Neural Processes , 2019, ICLR.

[9]  Kirthevasan Kandasamy,et al.  High Dimensional Bayesian Optimisation and Bandits via Additive Models , 2015, ICML.

[10]  Neil D. Lawrence,et al.  Batch Bayesian Optimization via Local Penalization , 2015, AISTATS.

[11]  Matthias Poloczek,et al.  A Framework for Bayesian Optimization in Embedded Subspaces , 2019, ICML.

[12]  Nando de Freitas,et al.  Bayesian Optimization in a Billion Dimensions via Random Embeddings , 2013, J. Artif. Intell. Res..

[13]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[14]  Aaron Klein,et al.  Bayesian Optimization with Robust Bayesian Neural Networks , 2016, NIPS.

[15]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[16]  Max Welling,et al.  BOCK : Bayesian Optimization with Cylindrical Kernels , 2018, ICML.

[17]  Andreas Krause,et al.  Parallelizing Exploration-Exploitation Tradeoffs with Gaussian Process Bandit Optimization , 2012, ICML.

[18]  Michalis K. Titsias,et al.  Variational Learning of Inducing Variables in Sparse Gaussian Processes , 2009, AISTATS.

[19]  Neil D. Lawrence,et al.  Structured Variationally Auto-encoded Optimization , 2018, ICML.

[20]  Ariel D. Procaccia,et al.  Variational Dropout and the Local Reparameterization Trick , 2015, NIPS.

[21]  Cheng Li,et al.  Budgeted Batch Bayesian Optimization , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[22]  Ali Jalali,et al.  Hybrid Batch Bayesian Optimization , 2012, ICML.

[23]  Andrew Gordon Wilson,et al.  GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration , 2018, NeurIPS.

[24]  Svetha Venkatesh,et al.  Batch Bayesian optimization using multi-scale search , 2020, Knowl. Based Syst..

[25]  新 雅夫,et al.  ASHRAE(American Society of Heating,Refrigerating and Air-Conditioning Engineers)大会"国際年"行事に参加して , 1975 .

[26]  Vikas Chandan,et al.  Physics-constrained Deep Learning of Multi-zone Building Thermal Dynamics , 2020, Energy and Buildings.

[27]  Torsten Hoefler,et al.  The digital revolution of Earth-system science , 2021, Nature Computational Science.

[28]  Thomas P. Parnell,et al.  Sampling Acquisition Functions for Batch Bayesian Optimization , 2019, ArXiv.