Deep Learning to Estimate Permeability using Geophysical Data

Time-lapse electrical resistivity tomography (ERT) is a popular geophysical method to estimate three-dimensional (3D) permeability fields from electrical potential difference measurements. Traditional inversion and data assimilation methods are used to ingest this ERT data into hydrogeophysical models to estimate permeability. Due to ill-posedness and the curse of dimensionality, existing inversion strategies provide poor estimates and low resolution of the 3D permeability field. Recent advances in deep learning provide us with powerful algorithms to overcome this challenge. This paper presents a deep learning (DL) framework to estimate the 3D subsurface permeability from time-lapse ERT data. To test the feasibility of the proposed framework, we train DL-enabled inverse models on simulation data. Subsurface process models based on hydrogeophysics are used to generate this synthetic data for deep learning analyses. Training performed on limited simulation data resulted in the DL model overfitting. An advanced data augmentation based on mixup is implemented to generate additional training samples to overcome this issue. This mixup technique creates weakly labeled (low-fidelity) samples from strongly labeled (high-fidelity) data. The weakly labeled training data is then used to develop DL-enabled inverse models and reduce over-fitting. As the data samples are from a high-dimensional space, unsupervised learning based on principal component analysis is employed to reduce dimensionality. A deep neural network is then trained to map the encoded ERT to encoded permeability. This mixup training and unsupervised learning allowed us to build a fast and reasonably accurate DL-based inverse model under limited simulation data. Results show that proposed weak supervised learning can capture salient spatial features in the 3D permeability field. Quantitatively, the average mean squared error (in terms of the natural log) on the strongly labeled training, validation, and test datasets is less than 0.5. The R2-score (global metric) is greater than 0.75, and the percent error in each cell (local metric) is less than 10%. Finally, an added benefit in terms of computational cost is that the proposed DL-based inverse model is at least O(104) times faster than running a forward model. Note that traditional inversion may require multiple forward model simulations (e.g., in the order of 10 to 1000), which are very expensive. This computational savings ( ≈ O(105)−O(107) ) makes the proposed DL-based inverse model attractive for subsurface imaging and real-time ERT monitoring applications due to fast and yet reasonably accurate estimations of permeability field.

[1]  D. Siler,et al.  3-D Geologic Controls of Hydrothermal Fluid Flow at Brady geothermal field, Nevada, USA , 2021, Geothermics.

[2]  Estella A. Atekwana,et al.  Geophysical Signatures of Microbial Activity at Hydrocarbon Contaminated Sites: A Review , 2010 .

[3]  Tianfang Xu,et al.  Machine learning for hydrologic sciences: An introductory overview , 2021, WIREs Water.

[4]  L. Y. Hu,et al.  Multiple‐point geostatistics for modeling subsurface heterogeneity: A comprehensive review , 2008 .

[5]  Tian-Chyi J. Yeh,et al.  Applied Stochastic Hydrogeology. , 2005 .

[6]  Roelof Versteeg,et al.  Improved hydrogeophysical characterization and monitoring through parallel modeling and inversion of time-domain resistivity andinduced-polarization data , 2010 .

[7]  K. Singha,et al.  Advances in interpretation of subsurface processes with time‐lapse electrical imaging , 2015 .

[8]  Chao Yang,et al.  A Survey on Deep Transfer Learning , 2018, ICANN.

[9]  R. Guérin Borehole and surface-based hydrogeophysics , 2005 .

[10]  Estella A. Atekwana,et al.  Biogeophysics: A new frontier in Earth science research , 2009 .

[11]  Ponnuthurai N. Suganthan,et al.  Ensemble deep learning: A review , 2021, Engineering applications of artificial intelligence.

[12]  Hyun Oh Song,et al.  Puzzle Mix: Exploiting Saliency and Local Statistics for Optimal Mixup , 2020, ICML.

[13]  G E Hammond,et al.  Evaluating the performance of parallel subsurface simulators: An illustrative example with PFLOTRAN , 2014, Water resources research.

[14]  Clifford H. Thurber,et al.  Parameter estimation and inverse problems , 2005 .

[15]  H. S. Viswanathan,et al.  Understanding hydraulic fracturing: a multi-scale problem , 2016, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[16]  Xuesong Zhang,et al.  Integrating field observations and process-based modeling to predict watershed water quality under environmental perturbations , 2020 .

[17]  Jef Caers,et al.  Modeling Uncertainty in the Earth Sciences , 2011 .

[18]  Olaf Kolditz,et al.  A comparative simulation study of coupled THM processes and their effect on fractured rock permeability around nuclear waste repositories , 2009 .

[19]  H. Viswanathan,et al.  PFLOTRAN-SIP: A PFLOTRAN Module for Simulating Spectral-Induced Polarization of Electrical Impedance Data , 2019, Energies.

[20]  Taghi M. Khoshgoftaar,et al.  A survey on Image Data Augmentation for Deep Learning , 2019, Journal of Big Data.

[21]  B. Berkowitz Characterizing flow and transport in fractured geological media: A review , 2002 .

[22]  Joaquin Vanschoren,et al.  Meta-Learning: A Survey , 2018, Automated Machine Learning.

[23]  Andrew M. Stuart,et al.  Inverse problems: A Bayesian perspective , 2010, Acta Numerica.

[24]  Bahaa E. A. Saleh Introduction to Subsurface Imaging , 2011 .

[25]  Mrinal K. Sen,et al.  Global Optimization Methods in Geophysical Inversion , 1995 .

[26]  Skipper Seabold,et al.  Statsmodels: Econometric and Statistical Modeling with Python , 2010, SciPy.

[27]  Martin S. Fridson,et al.  Trends , 1948, Bankmagazin.

[28]  W. Marsden I and J , 2012 .

[29]  E. Woo,et al.  Nonlinear Inverse Problems in Imaging , 2012 .

[30]  Jeffrey L. Anderson,et al.  The Data Assimilation Research Testbed: A Community Facility , 2009 .

[31]  Zhun Deng,et al.  How Does Mixup Help With Robustness and Generalization? , 2020, ArXiv.

[32]  Venkat Lakshmi,et al.  Advancing process‐based watershed hydrological research using near‐surface geophysics: a vision for, and review of, electrical and magnetic geophysical methods , 2008 .

[33]  Satish Karra,et al.  Unsupervised Machine Learning Based on Non-Negative Tensor Factorization for Analyzing Reactive-Mixing , 2018, J. Comput. Phys..

[34]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[35]  Deep Learning for the Earth Sciences , 2021 .

[36]  Quanming Yao,et al.  Few-shot Learning: A Survey , 2019, ArXiv.

[37]  Xingyuan Chen,et al.  PFLOTRAN-E4D: A parallel open source PFLOTRAN module for simulating time-lapse electrical resistivity data , 2017, Comput. Geosci..

[38]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[39]  R. Hunt,et al.  Approaches to highly parameterized inversion-A guide to using PEST for groundwater-model calibration , 2010 .

[40]  Danna Zhou,et al.  d. , 1840, Microbial pathogenesis.

[41]  Hydrogeophysics , 2021, Encyclopedia of Geology.

[42]  G. Karniadakis,et al.  Deep transfer learning and data augmentation improve glucose levels prediction in type 2 diabetes patients , 2021, npj Digital Medicine.

[43]  Cédric Jamet,et al.  Data Assimilation Methods , 2013 .

[44]  Susan S. Hubbard,et al.  The emergence of hydrogeophysics for improved understanding of subsurface processes over multiple scales , 2015, Water resources research.

[45]  Hari S Viswanathan,et al.  A system model for geologic sequestration of carbon dioxide. , 2009, Environmental science & technology.

[46]  Steven Euijong Whang,et al.  MixRL: Data Mixing Augmentation for Regression using Reinforcement Learning , 2021, ArXiv.

[47]  Oliver J. Maclaren,et al.  What can be estimated? Identifiability, estimability, causal inference and ill-posed inverse problems , 2019, ArXiv.

[48]  Ioannis Mitliagkas,et al.  Manifold Mixup: Encouraging Meaningful On-Manifold Interpolation as a Regularizer , 2018, ArXiv.

[49]  Satish Karra,et al.  Shale gas and non-aqueous fracturing fluids: Opportunities and challenges for supercritical CO2 , 2015 .

[50]  Joonhong Ahn,et al.  Geological repository systems for safe disposal of spent nuclear fuels and radioactive waste , 2010 .

[51]  Taufiquar Khan,et al.  Physically based regularization of hydrogeophysical inverse problems for improved imaging of process‐driven systems , 2013 .

[52]  V. Vesselinov,et al.  Machine learning to identify geologic factors associated with production in geothermal fields: a case-study using 3D geologic data, Brady geothermal field, Nevada , 2021, Geothermal Energy.

[53]  S. P. Neuman,et al.  Trends, prospects and challenges in quantifying flow and transport through fractured rocks , 2005 .

[54]  Francisco Herrera,et al.  Data Preprocessing in Data Mining , 2014, Intelligent Systems Reference Library.

[55]  P. Shuai,et al.  Estimating Watershed Subsurface Permeability From Stream Discharge Data Using Deep Neural Networks , 2021, Frontiers in Earth Science.

[56]  L. Slater Near Surface Electrical Characterization of Hydraulic Conductivity: From Petrophysical Properties to Aquifer Geometries—A Review , 2007 .

[57]  R. G. Cummings,et al.  Mining earth's heat: hot dry rock geothermal energy , 1979 .

[58]  Hari S. Viswanathan,et al.  The cross-scale science of CO2 capture and storage: from pore scale to regional scale , 2012 .

[59]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[60]  S. Karra,et al.  Machine learning to discover mineral trapping signatures due to CO2 injection , 2021, International Journal of Greenhouse Gas Control.

[61]  Satish Karra,et al.  PFLOTRAN User Manual A Massively Parallel Reactive Flow and Transport Model for Describing Surface and Subsurface Processes , 2015 .

[62]  Zunlei Feng,et al.  Neural Style Transfer: A Review , 2017, IEEE Transactions on Visualization and Computer Graphics.

[63]  Satish Karra,et al.  AdjointNet: Constraining machine learning models with physics-based codes , 2021, ArXiv.

[64]  Machine Learning for Subsurface Characterization , 2020 .

[65]  Alexander Y. Sun,et al.  How can Big Data and machine learning benefit environment and water management: a survey of methods, applications, and future directions , 2019, Environmental Research Letters.

[66]  A. Scheuermann,et al.  Comparison of invasive and non-invasive electromagnetic methods in soil water content estimation of a dike model , 2009 .

[67]  Xingyuan Chen,et al.  DART-PFLOTRAN: An ensemble-based data assimilation system for estimating subsurface flow and transport model parameters , 2021, Environ. Model. Softw..

[68]  Glenn E. Hammond,et al.  Application of ensemble‐based data assimilation techniques for aquifer characterization using tracer data at Hanford 300 area , 2013 .

[69]  E. Atekwana,et al.  The Microbial Community Structure in Petroleum-Contaminated Sediments Corresponds to Geophysical Signatures , 2007, Applied and Environmental Microbiology.

[70]  K. B. Nakshatrala,et al.  A deep learning modeling framework to capture mixing patterns in reactive-transport systems , 2021, Communications in Computational Physics.

[71]  Chuan Lu,et al.  PFLOTRAN: Reactive Flow & Transport Code for Use on Laptops to Leadership-Class Supercomputers , 2012 .

[72]  M. Sahimi,et al.  Machine learning in geo- and environmental sciences: From small to large scale , 2020, Advances in Water Resources.

[73]  Tian Zhang,et al.  Understanding Mixup Training Methods , 2018, IEEE Access.

[74]  Samuli Siltanen,et al.  Linear and Nonlinear Inverse Problems with Practical Applications , 2012, Computational science and engineering.

[75]  Albert Tarantola,et al.  Inverse problem theory - and methods for model parameter estimation , 2004 .

[76]  B. Minsley,et al.  Multiscale geophysical imaging of the critical zone , 2015 .

[77]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[78]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[79]  Ognjen Grujic,et al.  Quantifying Uncertainty in Subsurface Systems , 2018, Quantifying Uncertainty in Subsurface Systems.

[80]  Gopinath Chennupati,et al.  On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networks , 2019, NeurIPS.

[81]  Timothy J. Johnson,et al.  Four‐dimensional electrical conductivity monitoring of stage‐driven river water intrusion: Accounting for water table effects using a transient mesh boundary and conditional inversion constraints , 2015 .

[82]  Kamyar Azizzadenesheli,et al.  Fourier Neural Operator for Parametric Partial Differential Equations , 2021, ICLR.

[83]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[84]  Michael S. Eldred,et al.  DAKOTA , A Multilevel Parallel Object-Oriented Framework for Design Optimization , Parameter Estimation , Uncertainty Quantification , and Sensitivity Analysis Version 4 . 0 User ’ s Manual , 2006 .

[85]  Adrian E. Roitberg,et al.  Less is more: sampling chemical space with active learning , 2018, The Journal of chemical physics.

[86]  Alexander Y. Sun,et al.  Discovering State‐Parameter Mappings in Subsurface Models Using Generative Adversarial Networks , 2018, Geophysical Research Letters.