Resolution-independent generative models based on operator learning for physics-constrained Bayesian inverse problems

The Bayesian inference approach is widely used to tackle inverse problems due to its versatile and natural ability to handle ill-posedness. However, it often faces challenges when dealing with situations involving continuous fields or large-resolution discrete representations (high-dimensional). Moreover, the prior distribution of unknown parameters is commonly difficult to be determined. In this study, an Operator Learning-based Generative Adversarial Network (OL-GAN) is proposed and integrated into the Bayesian inference framework to handle these issues. Unlike most Bayesian approaches, the distinctive characteristic of the proposed method is to learn the joint distribution of parameters and responses. By leveraging the trained generative model, the posteriors of the unknown parameters can theoretically be approximated by any sampling algorithm (e.g., Markov Chain Monte Carlo, MCMC) in a low-dimensional latent space shared by the components of the joint distribution. The latent space is typically a simple and easy-to-sample distribution (e.g., Gaussian, uniform), which significantly reduces the computational cost associated with the Bayesian inference while avoiding prior selection concerns. Furthermore, incorporating operator learning enables resolution-independent in the generator. Predictions can be obtained at desired coordinates, and inversions can be performed even if the observation data are misaligned with the training data. Finally, the effectiveness of the proposed method is validated through several numerical experiments.

[1]  Steven G. Johnson,et al.  Multifidelity deep neural operators for efficient learning of partial differential equations with application to fast inverse design of nanoscale heat transport , 2022, Physical Review Research.

[2]  Hu Wang,et al.  Data-driven inverse method with uncertainties for path parameters of variable stiffness composite laminates , 2022, Structural and Multidisciplinary Optimization.

[3]  G. Karniadakis,et al.  A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data , 2021, Computer Methods in Applied Mechanics and Engineering.

[4]  Eric Darve,et al.  Solving inverse problems in stochastic models using deep neural networks and adversarial training , 2021 .

[5]  Dhruv V. Patel,et al.  Solution of Physics-based Bayesian Inverse Problems with Deep Generative Priors , 2021, Computer Methods in Applied Mechanics and Engineering.

[6]  George Em Karniadakis,et al.  Learning Functional Priors and Posteriors from Data and Physics , 2021, J. Comput. Phys..

[7]  Assad A. Oberai,et al.  GAN-Based Priors for Quantifying Uncertainty in Supervised Learning , 2021, SIAM/ASA J. Uncertain. Quantification.

[8]  Jaimit Parikh,et al.  Integration of AI and mechanistic modeling in generative adversarial networks for stochastic inverse problems , 2020, ArXiv.

[9]  James E. Warner,et al.  Inverse Estimation of Elastic Modulus Using Physics-Informed Generative Adversarial Networks , 2020, ArXiv.

[10]  Liu Yang,et al.  B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Data , 2020, J. Comput. Phys..

[11]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[12]  George Em Karniadakis,et al.  Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators , 2019, Nature Machine Intelligence.

[13]  Hu Wang,et al.  A novel adaptive approximate Bayesian computation method for inverse heat conduction problem , 2019, International Journal of Heat and Mass Transfer.

[14]  Hu Wang,et al.  A new POD-based approximate bayesian computation method to identify parameters for formed AHSS , 2019, International Journal of Solids and Structures.

[15]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[16]  Liu Yang,et al.  Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations , 2018, SIAM J. Sci. Comput..

[17]  Jens Behrmann,et al.  Invertible Residual Networks , 2018, ICML.

[18]  Alexander A. Alemi,et al.  Uncertainty in the Variational Information Bottleneck , 2018, ArXiv.

[19]  Daniela Calvetti,et al.  Inverse problems: From regularization to Bayesian inference , 2018 .

[20]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[21]  Vincent Dumoulin,et al.  Generative Adversarial Networks: An Overview , 2017, 1710.07035.

[22]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[23]  John Salvatier,et al.  Probabilistic programming in Python using PyMC3 , 2016, PeerJ Comput. Sci..

[24]  Marco Cuturi,et al.  On Wasserstein Two-Sample Testing and Related Families of Nonparametric Tests , 2015, Entropy.

[25]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[26]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[27]  Radford M. Neal MCMC Using Hamiltonian Dynamics , 2011, 1206.1901.

[28]  John K Kruschke,et al.  Bayesian data analysis. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[29]  David Rubin,et al.  Introduction to Continuum Mechanics , 2009 .

[30]  Habib N. Najm,et al.  Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problems , 2008, J. Comput. Phys..

[31]  Habib N. Najm,et al.  Stochastic spectral methods for efficient Bayesian solution of inverse problems , 2005, J. Comput. Phys..

[32]  Nicholas Zabaras,et al.  A Bayesian inference approach to the inverse heat conduction problem , 2004 .

[33]  P. Stark Inverse problems as statistics , 2002 .

[34]  A. Maćkiewicz,et al.  Principal Components Analysis (PCA) , 1993 .

[35]  Limin Wang,et al.  Karhunen-Loeve expansions and their applications , 2008 .

[36]  Clifford H. Thurber,et al.  Parameter estimation and inverse problems , 2005 .

[37]  Siddhartha Chib,et al.  MARKOV CHAIN MONTE CARLO METHODS: COMPUTATION AND INFERENCE , 2001 .

[38]  C. Vogel Computational Methods for Inverse Problems , 1987 .