Deep Bayesian inference for seismic imaging with tasks

We propose to use techniques from Bayesian inference and deep neural networks to translate uncertainty in seismic imaging to uncertainty in tasks performed on the image, such as horizon tracking. Seismic imaging is an ill-posed inverse problem because of bandwidth and aperture limitations, which is hampered by the presence of noise and linearization errors. Many regularization methods, such as transform-domain sparsity promotion, have been designed to deal with the adverse effects of these errors, however, these methods run the risk of biasing the solution and do not provide information on uncertainty in the image space and how this uncertainty impacts certain tasks on the image. A systematic approach is proposed to translate uncertainty due to noise in the data to confidence intervals of automatically tracked horizons in the image. The uncertainty is characterized by a convolutional neural network (CNN) and to assess these uncertainties, samples are drawn from the posterior distribution of the CNN weights, used to parameterize the image. Compared to traditional priors, it is argued in the literature that these CNNs introduce a flexible inductive bias that is a surprisingly good fit for a diverse set of problems, including medical imaging, compressive sensing, and diffraction tomography. The method of stochastic gradient Langevin dynamics is employed to sample from the posterior distribution. This method is designed to handle large scale Bayesian inference problems with computationally expensive forward operators as in seismic imaging. Aside from offering a robust alternative to maximum a posteriori estimate that is prone to overfitting, access to these samples allow us to translate uncertainty in the image, due to noise in the data, to uncertainty on the tracked horizons. For instance, it admits estimates for the pointwise standard deviation on the image and for confidence intervals on its automatically tracked horizons.

[1]  S. Arridge,et al.  Unsupervised knowledge-transfer for learned image reconstruction , 2021, Inverse problems.

[2]  James Theiler,et al.  Physics-Consistent Data-Driven Waveform Inversion With Adaptive Data Augmentation , 2020, IEEE Geoscience and Remote Sensing Letters.

[3]  Fantong Kong,et al.  Deep Prior-Based Unsupervised Reconstruction of Irregularly Sampled Seismic Data , 2022, IEEE Geoscience and Remote Sensing Letters.

[4]  Carola-Bibiane Schönlieb,et al.  End-to-end reconstruction meets data-driven regularization for inverse problems , 2021, NeurIPS.

[5]  Xiangliang Zhang,et al.  Mapping full seismic waveforms to vertical velocity profiles by deep learning , 2021, GEOPHYSICS.

[6]  Yongchae Cho,et al.  Estimation and uncertainty analysis of the CO2 storage volume in the sleipner field via 4D reversible-jump markov-chain Monte Carlo , 2021 .

[7]  Gerrit Blacquière,et al.  Source deghosting of coarsely sampled common-receiver data using a convolutional neural network , 2021 .

[8]  Rajiv Kumar,et al.  Low-rank representation of omnidirectional subsurface extended image volumes , 2021 .

[9]  Felix J. Herrmann,et al.  Learning by example: fast reliability-aware seismic imaging with normalizing flows , 2021, First International Meeting for Applied Geoscience & Energy Expanded Abstracts.

[10]  E. Verschuur,et al.  Training deep networks with only synthetic data: Deep-learning-based near-offset reconstruction for (closed-loop) surface-related multiple estimation on shallow-water field data , 2021 .

[11]  Felix J. Herrmann,et al.  Preconditioned training of normalizing flows for variational inference in inverse problems , 2021, ArXiv.

[12]  Nikola B. Kovachki,et al.  Fourier Neural Operator for Parametric Partial Differential Equations , 2020, ICLR.

[13]  Lihua Fu,et al.  Deep-seismic-prior-based reconstruction of seismic data using convolutional neural networks , 2019, GEOPHYSICS.

[14]  Ullrich Köthe,et al.  HINT: Hierarchical Invertible Neural Transport for Density Estimation and Bayesian Inference , 2019, AAAI.

[15]  Alexander Schlaefer,et al.  A Mean-Field Variational Inference Approach to Deep Image Prior for Inverse Problems in Medical Imaging , 2021, MIDL.

[16]  Rajiv Kumar,et al.  Enabling uncertainty quantification for seismic data preprocessing using normalizing flows (NF) – An interpolation example , 2021, First International Meeting for Applied Geoscience & Energy Expanded Abstracts.

[17]  A. Curtis,et al.  Bayesian Seismic Tomography using Normalizing Flows , 2020, Geophysical Journal International.

[18]  F. Liang,et al.  A Contour Stochastic Gradient Langevin Dynamics Algorithm for Simulations of Multi-modal Distributions , 2020, NeurIPS.

[19]  Laurent Demanet,et al.  Quality control of deep generator priors for statistical seismic inverse problems , 2020 .

[20]  Youssef M. Marzouk,et al.  Bayesian seismic inversion: Measuring Langevin MCMC sample quality with kernels , 2020 .

[21]  Andreas Hauptmann,et al.  Deep learning in photoacoustic tomography: current approaches and future directions , 2020, Journal of Biomedical Optics.

[22]  Gregory Ely,et al.  Uncertainty quantification in time-lapse seismic imaging: a full-waveform approach , 2020 .

[23]  S. Purves,et al.  Deep Bayesian Neural Networks for Fault Identification and Uncertainty Quantification , 2020, First EAGE Digitalization Conference and Exhibition.

[24]  Felix J. Herrmann,et al.  Faster Uncertainty Quantification for Inverse Problems with Conditional Normalizing Flows , 2020, ArXiv.

[25]  S. Fomel,et al.  Deep learning for relative geologic time and seismic horizons , 2020, GEOPHYSICS.

[26]  Sergey Fomel,et al.  Waveform embedding: Automatic horizon picking with unsupervised deep learning , 2020 .

[27]  Nikola B. Kovachki,et al.  Conditional Sampling With Monotone GANs , 2020, ArXiv.

[28]  Nam Pham,et al.  Improving the resolution of migrated images by approximating the inverse Hessian using deep learning , 2020, GEOPHYSICS.

[29]  Alexandros G. Dimakis,et al.  Deep Learning Techniques for Inverse Problems in Imaging , 2020, IEEE Journal on Selected Areas in Information Theory.

[30]  Felix J. Herrmann,et al.  Parameterizing uncertainty by deep invertible networks, an application to reservoir characterization , 2020, SEG Technical Program Expanded Abstracts 2020.

[31]  Felix J. Herrmann,et al.  Transfer learning in large-scale ocean bottom seismic wavefield reconstruction , 2020, SEG Technical Program Expanded Abstracts 2020.

[32]  F. Herrmann,et al.  Weak deep priors for seismic imaging , 2020, SEG Technical Program Expanded Abstracts 2020.

[33]  Aaron Defazio,et al.  End-to-End Variational Networks for Accelerated MRI Reconstruction , 2020, MICCAI.

[34]  Felix J. Herrmann,et al.  Uncertainty quantification in imaging and automatic horizon tracking: a Bayesian deep-prior based approach , 2020, SEG Technical Program Expanded Abstracts 2020.

[35]  Bas Peters,et al.  Fully reversible neural networks for large-scale 3D seismic horizon tracking , 2020 .

[36]  Pavel Izmailov,et al.  Bayesian Deep Learning and a Probabilistic Perspective of Generalization , 2020, NeurIPS.

[37]  Yisong Yue,et al.  On the distance between two neural networks and the stability of learning , 2020, NeurIPS.

[38]  Sergey Fomel,et al.  Deep learning parameterization for geophysical inverse problems , 2020 .

[39]  Felix J. Herrmann,et al.  A deep-learning based Bayesian approach to seismic imaging and uncertainty quantification , 2020, EAGE 2020 Annual Conference & Exhibition Online.

[40]  Laurent Demanet,et al.  Extrapolated full-waveform inversion with deep learning , 2019, GEOPHYSICS.

[41]  Xin Zhang,et al.  Seismic Tomography Using Variational Inference Methods , 2019, Journal of Geophysical Research: Solid Earth.

[42]  Ali Ahmed,et al.  Invertible generative models for inverse problems: mitigating representation error and dataset bias , 2019, ICML.

[43]  Charles Yount,et al.  Architecture and Performance of Devito, a System for Automated Stencil Computation , 2018, ACM Trans. Math. Softw..

[44]  Laurent Demanet,et al.  Elastic full-waveform inversion with extrapolated low-frequency data , 2016, SEG Technical Program Expanded Abstracts 2020.

[45]  L. Demanet,et al.  Deep generator priors for Bayesian seismic inversion , 2020 .

[46]  Yue Wu,et al.  InversionNet: An Efficient and Accurate Data-Driven Full Waveform Inversion , 2020, IEEE Transactions on Computational Imaging.

[47]  Patrick Putzky,et al.  Invert to Learn to Invert , 2019, NeurIPS.

[48]  George A. McMechan,et al.  Parametric convolutional neural network-domain full-waveform inversion , 2019, GEOPHYSICS.

[49]  P. Guo,et al.  Bayesian transdimensional seismic full-waveform inversion with a dipping layer parameterization , 2019, GEOPHYSICS.

[50]  Felix J. Herrmann,et al.  The importance of transfer learning in seismic modeling and imaging , 2019, GEOPHYSICS.

[51]  Felipe Pereira,et al.  A two-stage Markov chain Monte Carlo method for seismic inversion and uncertainty quantification , 2019, GEOPHYSICS.

[52]  Subhransu Maji,et al.  Shape Reconstruction Using Differentiable Projections and Deep Priors , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[53]  F. Herrmann,et al.  Learned imaging with constraints and uncertainty quantification , 2019, ArXiv.

[54]  T. Alkhalifah,et al.  Regularized elastic full-waveform inversion using deep learning , 2019, GEOPHYSICS.

[55]  Mrinal K. Sen,et al.  A gradient based MCMC method for FWI and uncertainty analysis , 2019, SEG Technical Program Expanded Abstracts 2019.

[56]  Andrew Gordon Wilson,et al.  Subspace Inference for Bayesian Deep Learning , 2019, UAI.

[57]  Simon R. Arridge,et al.  Solving inverse problems using data-driven models , 2019, Acta Numerica.

[58]  G. Schuster,et al.  Deep Convolutional Neural Network and Sparse Least Squares Migration. , 2019 .

[59]  Subhransu Maji,et al.  A Bayesian Perspective on the Deep Image Prior , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[60]  Daniel Kunin,et al.  Loss Landscapes of Regularized Linear Autoencoders , 2019, ICML.

[61]  Eldad Haber,et al.  Multi-resolution neural networks for tracking seismic horizons from few training images , 2018, Interpretation.

[62]  Regularization by Architecture: A Deep Prior Approach for Inverse Problems , 2018, Journal of Mathematical Imaging and Vision.

[63]  Barnabás Póczos,et al.  Gradient Descent Provably Optimizes Over-parameterized Neural Networks , 2018, ICLR.

[64]  Reinhard Heckel,et al.  Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networks , 2018, ICLR.

[65]  Martin J. Blunt,et al.  Stochastic Seismic Waveform Inversion Using Generative Adversarial Networks as a Geological Prior , 2018, Mathematical Geosciences.

[66]  Eric Moulines,et al.  The promises and pitfalls of Stochastic Gradient Langevin Dynamics , 2018, NeurIPS.

[67]  Jonas Adler,et al.  Deep Bayesian Inversion , 2018, ArXiv.

[68]  F. Herrmann,et al.  Uncertainty quantification for inverse problems with weak partial-differential-equation constraints , 2018, GEOPHYSICS.

[69]  Felix J. Herrmann,et al.  Interactive comment on “ Devito ( v 3 . 1 . 0 ) : an embedded domain-specific language for finite differences and geophysical exploration , 2018 .

[70]  Sergey Fomel,et al.  Least-squares horizons with local slopes and multigrid correlations , 2018, GEOPHYSICS.

[71]  R. Arnold,et al.  Interrogation theory , 2018, Geophysical Journal International.

[72]  R. Gibson,et al.  Seismic inversion and uncertainty quantification using transdimensional Markov chain Monte Carlo method , 2018, GEOPHYSICS.

[73]  Gregory Ely,et al.  Assessing uncertainties in velocity models and images with a fast nonlinear uncertainty quantification method , 2018 .

[74]  Hao Li,et al.  Visualizing the Loss Landscape of Neural Nets , 2017, NeurIPS.

[75]  Andrea Vedaldi,et al.  Deep Image Prior , 2017, International Journal of Computer Vision.

[76]  Sam Kaplan,et al.  Low frequency full waveform seismic inversion within a tree based Bayesian framework , 2018 .

[77]  Jianping Huang,et al.  Plane-wave least-squares reverse time migration with a preconditioned stochastic conjugate gradient method , 2018 .

[78]  Alexandros G. Dimakis,et al.  Compressed Sensing using Generative Models , 2017, ICML.

[79]  Matus Telgarsky,et al.  Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysis , 2017, COLT.

[80]  O. Ghattas,et al.  A Bayesian approach to estimate uncertainty for full-waveform inversion using a priori information from depth migration , 2016 .

[81]  Lawrence Carin,et al.  Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networks , 2015, AAAI.

[82]  Yee Whye Teh,et al.  Consistency and Fluctuations For Stochastic Gradient Langevin Dynamics , 2014, J. Mach. Learn. Res..

[83]  Xue Chen,et al.  L1 norm constrained migration of blended data with the FISTA algorithm , 2015 .

[84]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[85]  Shakir Mohamed,et al.  Variational Inference with Normalizing Flows , 2015, ICML.

[86]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[87]  Felix J. Herrmann,et al.  Fast imaging with surface-related multiples by sparse inversion , 2015 .

[88]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[89]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[90]  Robert L. Wolpert,et al.  Statistical Inference , 2019, Encyclopedia of Social Network Analysis and Mining.

[91]  Chong Zeng,et al.  Least-squares reverse time migration: Inversion-based imaging toward true reflectivity , 2014 .

[92]  Hiroshi Nakagawa,et al.  Approximation Analysis of Stochastic Gradient Langevin Dynamics by using Fokker-Planck Equation and Ito Process , 2014, ICML.

[93]  M. Burger,et al.  Maximum a posteriori estimates in linear inverse problems with log-concave priors are proper Bayes estimators , 2014, 1402.5297.

[94]  S. Dong,et al.  Least-squares reverse time migration: towards true amplitude imaging and improving the resolution , 2012 .

[95]  Eldad Haber,et al.  An Effective Method for Parameter Estimation with PDE Constraints with Multiple Right-Hand Sides , 2012, SIAM J. Optim..

[96]  Xiang Li,et al.  Efficient least‐squares imaging with sparsity promotion and compressive sensing , 2012 .

[97]  James Martin,et al.  A Stochastic Newton MCMC Method for Large-Scale Statistical Inverse Problems with Application to Seismic Inversion , 2012, SIAM J. Sci. Comput..

[98]  H. Crook,et al.  Building Complex Synthetic Models to Evaluate Acquisition Geometries and Velocity Inversion Technologies , 2012 .

[99]  Yee Whye Teh,et al.  Bayesian Learning via Stochastic Gradient Langevin Dynamics , 2011, ICML.

[100]  Felix J. Herrmann,et al.  Seismic Waveform Inversion by Stochastic Optimization , 2011 .

[101]  Radford M. Neal MCMC Using Hamiltonian Dynamics , 2011, 1206.1901.

[102]  John K Kruschke,et al.  Bayesian data analysis. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[103]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[104]  Alexander Shapiro,et al.  Stochastic Approximation approach to Stochastic Programming , 2013 .

[105]  Alejandro A. Valenciano Imaging by wave-equation inversion , 2008 .

[106]  H. Robbins A Stochastic Approximation Method , 1951 .

[107]  Tom M. Mitchell,et al.  The Need for Biases in Learning Generalizations , 2007 .

[108]  Alberto Malinverno,et al.  Two ways to quantify uncertainty in geophysical inverse problems , 2006 .

[109]  Albert Tarantola,et al.  Inverse problem theory - and methods for model parameter estimation , 2004 .

[110]  Alberto Malinverno,et al.  Expanded uncertainty quantification in inverse problems: Hierarchical Bayes and empirical Bayes , 2004 .

[111]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine Learning.

[112]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[113]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[114]  A. Malinverno Parsimonious Bayesian Markov chain Monte Carlo inversion in a nonlinear geophysical problem , 2002 .

[115]  A. Curtis,et al.  Prior information, sampling distributions, and the curse of dimensionality , 2001 .

[116]  G. Schuster,et al.  Least-squares migration of incomplete reflection data , 1999 .

[117]  Petros G. Voulgaris,et al.  On optimal ℓ∞ to ℓ∞ filtering , 1995, Autom..

[118]  Gerard T. Schuster,et al.  Least-Squares Cross-Well Migration , 1993 .

[119]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[120]  J. Virieux,et al.  Iterative asymptotic inversion in the acoustic approximation , 1992 .

[121]  Anders Krogh,et al.  A Simple Weight Decay Can Improve Generalization , 1991, NIPS.

[122]  A. Dawid Conditional Independence in Statistical Theory , 1979 .