Learning to regularize with a variational autoencoder for hydrologic inverse analysis

Inverse problems often involve matching observational data using a physical model that takes a large number of parameters as input. These problems tend to be under-constrained and require regularization to impose additional structure on the solution in parameter space. A central difficulty in regularization is turning a complex conceptual model of this additional structure into a functional mathematical form to be used in the inverse analysis. In this work we propose a method of regularization involving a machine learning technique known as a variational autoencoder (VAE). The VAE is trained to map a low-dimensional set of latent variables with a simple structure to the high-dimensional parameter space that has a complex structure. We train a VAE on unconditioned realizations of the parameters for a hydrological inverse problem. These unconditioned realizations neither rely on the observational data used to perform the inverse analysis nor require any forward runs of the physical model, thus making the computational cost of generating the training data minimal. The central benefit of this approach is that regularization is then performed on the latent variables from the VAE, which can be regularized simply. A second benefit of this approach is that the VAE reduces the number of variables in the optimization problem, thus making gradient-based optimization more computationally efficient when adjoint methods are unavailable. After performing regularization and optimization on the latent variables, the VAE then decodes the problem back to the original parameter space. Our approach constitutes a novel framework for regularization and optimization, readily applicable to a wide range of inverse problems. We call the approach RegAE.

[1]  Alan Edelman,et al.  Julia: A Fresh Approach to Numerical Computing , 2014, SIAM Rev..

[2]  Heinz-Otto Peitgen,et al.  The science of fractal images , 2011 .

[3]  L. Deng,et al.  The MNIST Database of Handwritten Digit Images for Machine Learning Research [Best of the Web] , 2012, IEEE Signal Processing Magazine.

[4]  Karl Kunisch,et al.  Total Generalized Variation , 2010, SIAM J. Imaging Sci..

[5]  Patrick Kofod Mogensen,et al.  Optim: A mathematical optimization package for Julia , 2018, J. Open Source Softw..

[6]  Velimir V. Vesselinov,et al.  Large‐scale inverse model analyses employing fast randomized data reduction , 2017 .

[7]  P. Kitanidis,et al.  Principal Component Geostatistical Approach for large-dimensional inverse problems , 2014, Water resources research.

[8]  George A. Zyvoloski,et al.  FEHMN 1.0: Finite element heat and mass transfer code , 1991 .

[9]  S. Kullback,et al.  Information Theory and Statistics , 1959 .

[10]  P. Mahalanobis On the generalized distance in statistics , 1936 .

[11]  LinLin Shen,et al.  Deep Feature Consistent Variational Autoencoder , 2016, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[12]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[13]  Satish Karra,et al.  PFLOTRAN User Manual A Massively Parallel Reactive Flow and Transport Model for Describing Surface and Subsurface Processes , 2015 .

[14]  Carl Doersch,et al.  Tutorial on Variational Autoencoders , 2016, ArXiv.

[15]  William W. Hager,et al.  A New Conjugate Gradient Method with Guaranteed Descent and an Efficient Line Search , 2005, SIAM J. Optim..

[16]  A Tikhonov,et al.  Solution of Incorrectly Formulated Problems and the Regularization Method , 1963 .

[17]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[18]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[19]  Peter K. Kitanidis,et al.  Large‐scale hydraulic tomography and joint inversion of head and tracer data using the Principal Component Geostatistical Approach (PCGA) , 2014 .

[20]  Daniel M. Tartakovsky,et al.  Linear functional minimization for inverse modeling , 2015 .

[21]  Julia Deniz Yuret Knet : beginning deep learning with 100 lines of , 2016 .

[22]  E. G. Vomvoris,et al.  A geostatistical approach to the inverse problem in groundwater modeling (steady state) and one‐dimensional simulations , 1983 .