Multicore Parallel Tempering Bayeslands for Basin and Landscape Evolution

In recent years, Bayesian inference has become a popular methodology for the estimation and uncertainty quantification of parameters in geological and geophysical forward models. Badlands is a basin and landscape evolution forward model for simulating topography evolution at a large range of spatial and time scales. Previously, Bayesian inference has been used for parameter estimation and uncertainty quantification in Badlands, an extension known as Bayeslands. It was demonstrated that the posterior surface of these parameters could exhibit highly irregular features such as multi-modality and discontinuities making standard Markov Chain Monte Carlo (MCMC) sampling difficult. Parallel tempering (PT) is an advanced MCMC method suited for irregular and multi-modal distributions. Moreover, PT is more suitable for multi-core implementations that can speed up computationally expensive geophysical models. In this paper, we present a multi-core PT algorithm implemented in a high performance computing architecture for enhancing Bayeslands. The results show that PT in Bayeslands not only reduces the computation time over a multi-core architecture, but also provides a means to improve the sampling process in a multi-modal landscape. This motivates its usage in larger-scale problems in basin and landscape evolution models.

[1]  Cajo J. F. ter Braak,et al.  A Markov Chain Monte Carlo version of the genetic algorithm Differential Evolution: easy Bayesian computing for real parameter spaces , 2006, Stat. Comput..

[2]  Jean Braun,et al.  Constraining the stream power law: a novel approach combining a landscape evolution model and an inversion method , 2013 .

[3]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[4]  F. Calvo,et al.  All-exchanges parallel tempering. , 2005, The Journal of chemical physics.

[5]  P. Rocca,et al.  Evolutionary optimization as applied to inverse scattering problems , 2009 .

[6]  Scott D. Brown,et al.  A simple introduction to Markov Chain Monte–Carlo sampling , 2016, Psychonomic bulletin & review.

[7]  A. Malinverno.  Parsimonious Bayesian Markov chain Monte Carlo inversion in a nonlinear geophysical problem , 2002 .

[8]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[9]  Tristan Salles,et al.  Combined hillslope diffusion and sediment transport simulation applied to landscape dynamics modelling , 2015 .

[10]  Eric Moulines,et al.  Adaptive parallel tempering algorithm , 2012, 1205.1076.

[11]  M. Girolami,et al.  Riemann manifold Langevin and Hamiltonian Monte Carlo methods , 2011, Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[12]  U. Hansmann.  Parallel tempering algorithm for conformational studies of biological molecules , 1997, physics/9710041.

[13]  D. Kofke,et al.  Selection of temperature intervals for parallel-tempering simulations. , 2005, The Journal of chemical physics.

[14]  M. J. Salinger.  New Zealand Climate: I. Precipitation Patterns , 1980 .

[15]  Jocelyn Chanussot,et al.  Challenges and Opportunities of Multimodality and Data Fusion in Remote Sensing , 2014, Proceedings of the IEEE.

[16]  J. Rejman,et al.  Spatial and temporal variations in erodibility of loess soil , 1998 .

[17]  Gilles Brocard,et al.  A unified framework for modelling sediment fate from source to sink and its interactions with reef systems over geological times , 2018, Scientific Reports.

[18]  S. Chib,et al.  Understanding the Metropolis-Hastings Algorithm , 1995 .

[19]  George Casella,et al.  A Short History of Markov Chain Monte Carlo: Subjective Recollections from Incomplete Data , 2008, 0808.2902.

[20]  Marie-Alice Harel,et al.  Global analysis of the stream power law parameters based on worldwide 10Be denudation rates , 2016 .

[21]  A. Tarantola.  Popper, Bayes and the inverse problem , 2006 .

[22]  L. R. Scott,et al.  Finite element interpolation of nonsmooth functions satisfying boundary conditions , 1990 .

[23]  Sebastiano Foti,et al.  A Monte Carlo multimodal inversion of surface waves , 2010 .

[24]  Radford M. Neal.  MCMC Using Hamiltonian Dynamics , 2011, 1206.1901.

[25]  Pierre Baldi,et al.  Gradient descent learning algorithm overview: a general dynamical systems perspective , 1995, IEEE Trans. Neural Networks.

[26]  Leslie Lamport,et al.  On interprocess communication , 1986, Distributed Computing.

[27]  H. Jeffreys.  An invariant form for the prior probability in estimation problems , 1946, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[28]  Nicole M. Gasparini,et al.  A generalized power law approximation for fluvial incision of bedrock channels , 2011 .

[29]  Dirk Thierens,et al.  Evolutionary Markov Chain Monte Carlo , 2003, Artificial Evolution.

[30]  Wolfgang Schwanghart,et al.  Accurate simulation of transient landscape evolution by eliminating numerical diffusion: the TTLEM 1.0 model , 2016 .

[31]  G. Tucker,et al.  Modelling landscape evolution , 2010 .

[32]  Andrew M. Stuart,et al.  Geometric MCMC for infinite-dimensional inverse problems , 2016, J. Comput. Phys..

[33]  Timothy A. Cross,et al.  Construction and Application of a Stratigraphic Inverse Model , 1999 .

[34]  Gilles Brocard,et al.  pyBadlands: A framework to simulate sediment transport, landscape dynamics and basin stratigraphic evolution through space and time , 2018, PloS one.

[35]  Mrinal K. Sen,et al.  Global Optimization Methods in Geophysical Inversion , 1995 .

[36]  Klaus Mosegaard,et al.  A SIMULATED ANNEALING APPROACH TO SEISMIC MODEL OPTIMIZATION WITH SPARSE PRIOR INFORMATION , 1991 .

[37]  Geoffrey C. Fox,et al.  Implementing an intervisibility analysis model on a parallel computing system , 1992 .

[38]  S. Fienberg.  When did Bayesian inference become "Bayesian"? , 2006 .

[39]  Caroline C. Ummenhofer,et al.  Interannual Extremes in New Zealand Precipitation Linked to Modes of Southern Hemisphere Climate Variability , 2007 .

[40]  Stephen E. Fienberg,et al.  When did Bayesian Inference Become , 1973 .

[41]  Tristan Salles,et al.  Badlands: A parallel basin and landscape dynamics model , 2016, SoftwareX.

[42]  Rohitash Chandra,et al.  Efficiency and robustness in Monte Carlo sampling of 3-D geophysical inversions with Obsidian v0.1.2: Setting up for success , 2018, Geoscientific Model Development.

[43]  Christos-Savvas Bouganis,et al.  Particle MCMC algorithms and architectures for accelerating inference in state-space models☆ , 2017, Int. J. Approx. Reason..

[44]  Sean D. Willett,et al.  Rock uplift and erosion rate history of the Bergell intrusion from the inversion of low temperature thermochronometric data , 2014 .

[45]  Navtej Singh,et al.  Parallel Astronomical Data Processing with Python: Recipes for multicore machines , 2013, Astron. Comput..

[46]  Nicholas J. White,et al.  Estimating uplift rate histories from river profiles using African examples , 2010 .

[47]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[48]  J. Morel,et al.  Landscape evolution models: A review of their fundamental equations , 2014 .

[49]  G. Parisi,et al.  Simulated tempering: a new Monte Carlo scheme , 1992, hep-lat/9205018.

[50]  Tristan Salles,et al.  Badlands: An open-source, flexible and parallel framework to study landscape dynamics , 2016, Comput. Geosci..

[51]  M. Sambridge,et al.  Markov chain Monte Carlo (MCMC) sampling methods to determine optimal models, model resolution and model choice for Earth Science problems , 2009 .

[52]  Fabio Tozeto Ramos,et al.  Bayesian Joint Inversions for the Exploration of Earth Resources , 2013, IJCAI.

[53]  Nicolas Flament,et al.  A review of observations and models of dynamic topography , 2013 .

[54]  R. Cattin,et al.  Numerical modelling of erosion processes in the Himalayas of Nepal: effects of spatial variations of rock strength and precipitation , 2006, Geological Society, London, Special Publications.

[55]  Daniel E. J. Hobley,et al.  Field calibration of sediment flux dependent river incision , 2011 .

[56]  Radford M. Neal.  Sampling from multimodal distributions using tempered transitions , 1996, Stat. Comput..

[57]  David A. Yuen,et al.  Toward an automated parallel computing environment for geosciences , 2007 .

[58]  Ratneel Vikash Deo,et al.  Langevin-gradient parallel tempering for Bayesian neural learning , 2018, Neurocomputing.

[59]  Nicole M. Gasparini,et al.  The Landlab v1.0 OverlandFlow component: a Python tool for computing shallow-water flow across watersheds , 2017 .

[60]  Mrinal K. Sen,et al.  Bayesian inference, Gibbs' sampler and uncertainty estimation in geophysical inversion , 1996 .

[61]  David R. Montgomery,et al.  Geologic constraints on bedrock river incision using the stream power law , 1999 .

[62]  Glenn Shafer,et al.  Belief Functions and Parametric Models , 1982, Classic Works of the Dempster-Shafer Theory of Belief Functions.

[63]  M. Menvielle,et al.  Bayesian inversion with Markov chains—I. The magnetotelluric one-dimensional case , 1999 .

[64]  Rohitash Chandra,et al.  BayesLands: A Bayesian inference approach for parameter uncertainty quantification in Badlands , 2018, Comput. Geosci..

[65]  Matthew Fox,et al.  Abrupt changes in the rate of Andean Plateau uplift from reversible jump Markov Chain Monte Carlo inversion of river profiles , 2014 .

[66]  D. van der Spoel,et al.  A temperature predictor for parallel tempering simulations. , 2008, Physical chemistry chemical physics : PCCP.

[67]  K. Hukushima,et al.  Exchange Monte Carlo Method and Application to Spin Glass Simulations , 1995, cond-mat/9512035.

[68]  Klaus Mosegaard,et al.  MONTE CARLO METHODS IN GEOPHYSICAL INVERSE PROBLEMS , 2002 .

[69]  William E. Dietrich,et al.  Modeling fluvial erosion on regional to continental scales , 1994 .

[70]  Jeffrey S. Rosenthal,et al.  Optimal Proposal Distributions and Adaptive MCMC , 2011 .

[71]  Malcolm Sambridge,et al.  A Parallel Tempering algorithm for probabilistic sampling and multimodal optimization , 2014 .

[72]  Peter M. A. Sloot,et al.  Application of parallel computing to stochastic parameter estimation in environmental models , 2006, Comput. Geosci..

[73]  Nicole M. Gasparini,et al.  Creative computing with Landlab: an open-source toolkit for building, coupling, and exploring two-dimensional numerical models of Earth-surface dynamics , 2016 .

[74]  Tristan Salles,et al.  Influence of mantle flow on the drainage of eastern Australia since the Jurassic Period , 2017 .

[75]  H. Haario,et al.  An adaptive Metropolis algorithm , 2001 .

[76]  Tom J. Coulthard,et al.  Landscape evolution models: a software review , 2001 .

[77]  Andrew Gelman,et al.  The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo , 2011, J. Mach. Learn. Res..

[78]  M. Sambridge.  Geophysical inversion with a neighbourhood algorithm—II. Appraising the ensemble , 1999 .

[79]  Arthur Gretton,et al.  Gradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families , 2015, NIPS.

[80]  Yaohang Li,et al.  A decentralized parallel implementation for parallel tempering algorithm , 2009, Parallel Comput..

[81]  Firas Hamze,et al.  High-performance Physics Simulations Using Multi-core CPUs and GPGPUs in a Volunteer Computing Context , 2011, Int. J. High Perform. Comput. Appl..

[82]  G. Tucker,et al.  Implications of sediment‐flux‐dependent river incision models for landscape evolution , 2002 .

[83]  Sean D. Willett,et al.  Tectonics from fluvial topography using formal linear inversion: Theory and applications to the Inyo Mountains, California , 2014 .

[84]  C. Geyer,et al.  Annealing Markov chain Monte Carlo with applications to ancestral inference , 1995 .

[85]  Niels Bohr,et al.  Monte Carlo sampling of solutions to inverse problems , 2004 .