Surrogate-assisted Bayesian inversion for landscape and basin evolution models

Abstract. The complex and computationally expensive nature of landscape evolution models poses significant challenges to the inference and optimization of unknown model parameters. Bayesian inference provides a methodology for estimation and uncertainty quantification of unknown model parameters. In our previous work, we developed parallel tempering Bayeslands as a framework for parameter estimation and uncertainty quantification for the Badlands landscape evolution model. Parallel tempering Bayeslands features high-performance computing that can feature dozens of processing cores running in parallel to enhance computational efficiency. Nevertheless, the procedure remains computationally challenging since thousands of samples need to be drawn and evaluated. In large-scale landscape evolution problems, a single model evaluation can take from several minutes to hours and in some instances, even days or weeks. Surrogate-assisted optimization has been used for several computationally expensive engineering problems which motivate its use in optimization and inference of complex geoscientific models. The use of surrogate models can speed up parallel tempering Bayeslands by developing computationally inexpensive models to mimic expensive ones. In this paper, we apply surrogate-assisted parallel tempering where the surrogate mimics a landscape evolution model by estimating the likelihood function from the model. We employ a neural-network-based surrogate model that learns from the history of samples generated. The entire framework is developed in a parallel computing infrastructure to take advantage of parallelism. The results show that the proposed methodology is effective in lowering the computational cost significantly while retaining the quality of model predictions.

[1]  M. Sambridge Geophysical inversion with a neighbourhood algorithm—II. Appraising the ensemble , 1999 .

[2]  Leslie Lamport,et al.  Interprocess Communication , 2020, Practical System Programming with C.

[3]  Radford M. Neal Sampling from multimodal distributions using tempered transitions , 1996, Stat. Comput..

[4]  Richard Scalzo,et al.  Bayesian geological and geophysical data fusion for the construction and uncertainty quantification of 3D geological models , 2021 .

[5]  Robert Will,et al.  Co-optimization of CO2-EOR and Storage Processes under Geological Uncertainty , 2017 .

[6]  Daniel E. J. Hobley,et al.  Field calibration of sediment flux dependent river incision , 2011 .

[7]  Zhengdong Lu,et al.  Fast neural network surrogates for very high dimensional physics-based models in computational oceanography , 2007, Neural Networks.

[8]  Rohitash Chandra,et al.  BayesReef: A Bayesian inference framework for modelling reef growth in response to environmental change and biological dynamics , 2018, Environ. Model. Softw..

[9]  Scott D. Brown,et al.  A simple introduction to Markov Chain Monte–Carlo sampling , 2016, Psychonomic bulletin & review.

[10]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[11]  Edgar Tello-Leal,et al.  A Review of Surrogate Assisted Multiobjective Evolutionary Algorithms , 2016, Comput. Intell. Neurosci..

[12]  Sebastian Scher,et al.  Toward Data‐Driven Weather and Climate Forecasting: Approximating a Simple General Circulation Model With Deep Learning , 2018, Geophysical Research Letters.

[13]  G. Tucker,et al.  Modelling landscape evolution , 2010 .

[14]  Bryan A. Tolson,et al.  Review of surrogate modeling in water resources , 2012 .

[15]  Rohitash Chandra,et al.  BayesLands: A Bayesian inference approach for parameter uncertainty quantification in Badlands , 2018, Comput. Geosci..

[16]  Cee Ing Teh,et al.  Reliability analysis of laterally loaded piles using response surface methods , 2000 .

[17]  D. van der Spoel,et al.  A temperature predictor for parallel tempering simulations. , 2008, Physical chemistry chemical physics : PCCP.

[18]  K. Hukushima,et al.  Exchange Monte Carlo Method and Application to Spin Glass Simulations , 1995, cond-mat/9512035.

[19]  Rohitash Chandra,et al.  Efficiency and robustness in Monte Carlo sampling of 3-D geophysical inversions with Obsidian v0.1.2: Setting up for success , 2018, Geoscientific Model Development.

[20]  William E. Dietrich,et al.  Modeling fluvial erosion on regional to continental scales , 1994 .

[21]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[22]  Jean Braun,et al.  A very efficient O(n), implicit and parallel method to solve the stream power equation governing fluvial incision and landscape evolution , 2013 .

[23]  Anthony J. Jakeman,et al.  A review of surrogate models and their application to groundwater modeling , 2015 .

[24]  G. Parisi,et al.  Simulated tempering: a new Monte Carlo scheme , 1992, hep-lat/9205018.

[25]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .

[26]  Tristan Salles,et al.  Badlands: An open-source, flexible and parallel framework to study landscape dynamics , 2016, Comput. Geosci..

[27]  A. Keane,et al.  Evolutionary Optimization of Computationally Expensive Problems via Surrogate Modeling , 2003 .

[28]  Gilles Brocard,et al.  pyBadlands: A framework to simulate sediment transport, landscape dynamics and basin stratigraphic evolution through space and time , 2018, PloS one.

[29]  Kok Wai Wong,et al.  Surrogate-Assisted Evolutionary Optimization Frameworks for High-Fidelity Engineering Design Problems , 2005 .

[30]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[31]  Wolfhard Janke,et al.  Make Life Simple: Unleash the Full Power of the Parallel Tempering Algorithm , 2008 .

[32]  H. Haario,et al.  An adaptive Metropolis algorithm , 2001 .

[33]  C. Geyer,et al.  Annealing Markov chain Monte Carlo with applications to ancestral inference , 1995 .

[34]  G. Tucker,et al.  Implications of sediment‐flux‐dependent river incision models for landscape evolution , 2002 .

[35]  Wolfgang Schwanghart,et al.  Accurate simulation of transient landscape evolution by eliminating numerical diffusion: the TTLEM 1.0 model , 2016 .

[36]  Rohitash Chandra,et al.  Multicore Parallel Tempering Bayeslands for Basin and Landscape Evolution , 2018, Geochemistry, Geophysics, Geosystems.

[37]  Ben Calderhead,et al.  A general construction for parallelizing Metropolis−Hastings algorithms , 2014, Proceedings of the National Academy of Sciences.

[38]  Bernhard Sendhoff,et al.  Generalizing Surrogate-Assisted Evolutionary Computation , 2010, IEEE Transactions on Evolutionary Computation.

[39]  Douglas C. Montgomery,et al.  Multiple response surface methods in computer simulation , 1977 .

[40]  Michael Gurnis,et al.  SPGM: A Scalable PaleoGeomorphology Model , 2018, SoftwareX.

[41]  Malcolm Sambridge,et al.  A Parallel Tempering algorithm for probabilistic sampling and multimodal optimization , 2014 .

[42]  Omar M. Knio,et al.  Surrogate-based parameter inference in debris flow model , 2018, Computational Geosciences.

[43]  Bradley P. Carlin,et al.  Markov Chain Monte Carlo in Practice: A Roundtable Discussion , 1998 .

[44]  I. Mandel,et al.  Dynamic temperature selection for parallel tempering in Markov chain Monte Carlo simulations , 2015, 1501.05823.

[45]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[46]  L. M. Berliner,et al.  Hierarchical Bayesian space-time models , 1998, Environmental and Ecological Statistics.

[47]  Rohitash Chandra,et al.  Surrogate-assisted parallel tempering for Bayesian neural learning , 2018, Eng. Appl. Artif. Intell..

[48]  F. Calvo,et al.  All-exchanges parallel tempering. , 2005, The Journal of chemical physics.

[49]  Raymond H. Myers,et al.  Response Surface Methods for Bi-Randomization Structures , 1996 .

[50]  Ratneel Vikash Deo,et al.  Langevin-gradient parallel tempering for Bayesian neural learning , 2018, Neurocomputing.

[51]  Nicole M. Gasparini,et al.  The Landlab v1.0 OverlandFlow component: a Python tool for computing shallow-water flow across watersheds , 2017 .

[52]  Léon Bottou,et al.  Large-Scale Machine Learning with Stochastic Gradient Descent , 2010, COMPSTAT.

[53]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[54]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[55]  Bradley P. Carlin,et al.  On MCMC sampling in hierarchical longitudinal models , 1999, Stat. Comput..

[56]  Andy J. Keane,et al.  Combining Global and Local Surrogate Models to Accelerate Evolutionary Optimization , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[57]  Leslie Lamport,et al.  On interprocess communication , 1986, Distributed Computing.

[58]  Yaochu Jin,et al.  Surrogate-assisted evolutionary computation: Recent advances and future challenges , 2011, Swarm Evol. Comput..

[59]  B. Berne,et al.  Replica exchange with solute tempering: a method for sampling biological systems in explicit water. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[60]  L. Bottou Stochastic Gradient Learning in Neural Networks , 1991 .