Uncertainty and interpretability analysis of encoder-decoder architecture for channel detection

We have adopted a method to understand uncertainty and interpretability of a Bayesian convolutional neural network for detecting 3D channel geobodies in seismic volumes. We measure heteroscedastic aleatoric uncertainty and epistemic uncertainty. Epistemic uncertainty captures the uncertainty of the network parameters, whereas heteroscedastic aleatoric uncertainty accounts for noise in the seismic volumes. We train a network modified from U-Net architecture on 3D synthetic seismic volumes, and then we apply it to field data. Tests on 3D field data sets from the Browse Basin, offshore Australia, and from Parihaka in New Zealand prove that uncertainty volumes are related to geologic uncertainty, model mispicks, and input noise. We analyze model interpretability on these data sets by creating saliency volumes with gradient-weighted class activation mapping. We find that the model takes a global-to-local approach to localize channel geobodies as well as the importance of different model components in overall strategy. Using channel probability, uncertainty, and saliency volumes, interpreters can accurately identify channel geobodies in 3D seismic volumes and also understand the model predictions.

[1]  David J. C. MacKay,et al.  A Practical Bayesian Framework for Backpropagation Networks , 1992, Neural Computation.

[2]  R. Lark,et al.  A statistical assessment of the uncertainty in a 3-D geological framework model , 2013 .

[3]  S. Fomel,et al.  SaltSeg: Automatic 3D salt segmentation using a deep convolutional neural network , 2019, Interpretation.

[4]  Alexandre Hoang Thiery,et al.  Uncertainty Quantification and Deep Ensembles , 2020, NeurIPS.

[5]  Quanshi Zhang,et al.  Visual interpretability for deep learning: a survey , 2018, Frontiers of Information Technology & Electronic Engineering.

[6]  Charles Blundell,et al.  Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles , 2016, NIPS.

[7]  Alex Kendall,et al.  What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? , 2017, NIPS.

[8]  Kumar Shridhar,et al.  Uncertainty Estimations by Softplus normalization in Bayesian Convolutional Neural Networks with Variational Inference , 2018 .

[9]  Xishuang Dong,et al.  A scalable deep learning platform for identifying geologic features from seismic attributes , 2017 .

[10]  Tomaso Poggio,et al.  Automated fault detection without seismic processing , 2017 .

[11]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Solomon Kullback,et al.  Information Theory and Statistics , 1960 .

[13]  Sepp Hochreiter,et al.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.

[14]  Sergey Fomel,et al.  FaultSeg3D: Using synthetic data sets to train an end-to-end convolutional neural network for 3D seismic fault segmentation , 2019, GEOPHYSICS.

[15]  Thomas Brox,et al.  Striving for Simplicity: The All Convolutional Net , 2014, ICLR.

[16]  A. Kiureghian,et al.  Aleatory or epistemic? Does it matter? , 2009 .

[17]  Lukas Mosser,et al.  Probabilistic Seismic Interpretation Using Bayesian Neural Networks , 2019 .

[18]  Vincent Dumoulin,et al.  Deconvolution and Checkerboard Artifacts , 2016 .

[19]  C. Bond Uncertainty in Structural Interpretation: Lessons to be learnt , 2015 .

[20]  A. Curtis,et al.  Dynamics of uncertainty in geological interpretation , 2010, Journal of the Geological Society.

[21]  Kurt J. Marfurt,et al.  Instantaneous spectral attributes to detect channels , 2007 .

[22]  Sergey Fomel,et al.  Poststack velocity analysis by separation and imaging of seismic diffractions , 2007 .

[23]  Nam Pham,et al.  Missing well log prediction using convolutional long short-term memory network , 2020 .

[24]  Mason Phillips,et al.  Plane-wave Sobel attribute for discontinuity enhancement in seismic images , 2017 .

[25]  Geoffrey E. Hinton,et al.  Keeping the neural networks simple by minimizing the description length of the weights , 1993, COLT '93.

[26]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[27]  Zoubin Ghahramani,et al.  Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference , 2015, ArXiv.

[28]  Albert Tarantola,et al.  Inverse problem theory - and methods for model parameter estimation , 2004 .

[29]  Ganapathy Krishnamurthi,et al.  Demystifying Brain Tumor Segmentation Networks: Interpretability and Uncertainty Analysis , 2020, Frontiers in Computational Neuroscience.

[30]  Roberto Cipolla,et al.  Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[31]  Xinming Wu Directional structure-tensor-based coherence to detect seismic faults and channels , 2017 .

[32]  Chong Wang,et al.  Stochastic variational inference , 2012, J. Mach. Learn. Res..

[33]  M. Jessell,et al.  Locating and quantifying geological uncertainty in three-dimensional models: Analysis of the Gippsland Basin, southeastern Australia , 2012 .

[34]  F. Horowitz,et al.  Towards incorporating uncertainty of structural data in 3D geological inversion , 2010 .

[35]  Nam Pham,et al.  Automatic channel detection using deep learning , 2019, Interpretation.

[36]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.

[37]  Yann LeCun,et al.  Transforming Neural-Net Output Levels to Probability Distributions , 1990, NIPS.

[38]  Gregory C. Beroza,et al.  Bayesian-Deep-Learning Estimation of Earthquake Location From Single-Station Observations , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[39]  Zoubin Ghahramani,et al.  Probabilistic machine learning and artificial intelligence , 2015, Nature.

[40]  Alex Kendall,et al.  Geometry and uncertainty in deep learning for computer vision , 2019 .

[41]  Alex Kendall,et al.  Concrete Dropout , 2017, NIPS.

[42]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .

[43]  Bruce S. Hart,et al.  Channel detection in 3-D seismic data using sweetness , 2008 .

[44]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[45]  Julien Cornebise,et al.  Weight Uncertainty in Neural Networks , 2015, ArXiv.

[46]  Aurèle Parriaux,et al.  Geological uncertainties associated with 3-D subsurface models , 2006, Comput. Geosci..

[47]  Anne H. Schistad Solberg,et al.  Convolutional neural networks for automated seismic interpretation , 2018, The Leading Edge.