Compressed Particle Methods for Expensive Models With Application in Astronomy and Remote Sensing

In many inference problems, the evaluation of complex and costly models is often required. In this context, Bayesian methods have become very popular in several fields over the last years, in order to obtain parameter inversion, model selection, or uncertainty quantification. Bayesian inference requires the approximation of complicated integrals involving (often costly) posterior distributions. Generally, this approximation is obtained by means of Monte–Carlo (MC) methods. In order to reduce the computational cost of the corresponding technique, surrogate models (also called emulators) are often employed. Another alternative approach is the so-called approximate Bayesian computation (ABC) scheme. ABC does not require the evaluation of the costly model but the ability to simulate artificial data according to that model. Moreover, in ABC, the choice of a suitable distance between real and artificial data is also required. In this work, we introduce a novel approach where the expensive model is evaluated only in some well-chosen samples. The selection of these nodes is based on the so-called compressed MC scheme. We provide theoretical results supporting the novel algorithms and give empirical evidence of the performance of the proposed method in several numerical experiments. Two of them are real-world applications in astronomy and satellite remote sensing. (Corresponding author: Luca Martino.)

[1]  Sonja Kuhnt,et al.  Design and analysis of computer experiments , 2010 .

[2]  Luca Martino,et al.  A review of multiple try MCMC algorithms for signal processing , 2017, Digit. Signal Process..

[3]  Michael A. West,et al.  Combined Parameter and State Estimation in Simulation-Based Filtering , 2001, Sequential Monte Carlo Methods in Practice.

[4]  Michael K. Pitt,et al.  Auxiliary Variable Based Particle Filters , 2001, Sequential Monte Carlo Methods in Practice.

[5]  Shuicheng Yan,et al.  Active learning with adaptive regularization , 2011, Pattern Recognit..

[6]  Paris Perdikaris,et al.  Machine learning of linear differential equations using Gaussian processes , 2017, J. Comput. Phys..

[7]  Tim Hesterberg,et al.  Monte Carlo Strategies in Scientific Computing , 2002, Technometrics.

[8]  Daniel Busby,et al.  Hierarchical adaptive experimental design for Gaussian process emulators , 2009, Reliab. Eng. Syst. Saf..

[9]  Luca Martino,et al.  Compressed Monte Carlo with application in particle filtering , 2021, Inf. Sci..

[10]  S. Qian,et al.  Observations and Orbital Analysis of the High-Amplitude Delta Scuti Star SZ Lyncis: The Unusual Orbital Precession , 2013 .

[11]  R. Plevin,et al.  Approximate Bayesian Computation in Evolution and Ecology , 2011 .

[12]  Simo Särkkä,et al.  Bayesian Filtering and Smoothing , 2013, Institute of Mathematical Statistics textbooks.

[13]  Piet Demeester,et al.  A Surrogate Modeling and Adaptive Sampling Toolbox for Computer Based Design , 2010, J. Mach. Learn. Res..

[14]  A. Doucet,et al.  Particle Markov chain Monte Carlo methods , 2010 .

[15]  Pau Closas,et al.  Importance Gaussian Quadrature , 2020, IEEE Transactions on Signal Processing.

[16]  Hoon Kim,et al.  Monte Carlo Statistical Methods , 2000, Technometrics.

[17]  Anders Sandberg,et al.  Feasibility of Whole Brain Emulation , 2011, PT-AI.

[18]  A. O'Hagan,et al.  Bayesian emulation of complex multi-output and dynamic computer models , 2010 .

[19]  Petar M. Djuric,et al.  Adaptive Importance Sampling: The past, the present, and the future , 2017, IEEE Signal Processing Magazine.

[20]  D. Montes,et al.  Chromospheric Activity and Orbital Solution of Six New Late-type Spectroscopic Binary Systems , 2006 .

[21]  S. Haykin,et al.  Cubature Kalman Filters , 2009, IEEE Transactions on Automatic Control.

[22]  R. Paul Butler,et al.  Two Jovian Planets around the Giant Star HD 202696: A Growing Population of Packed Massive Planetary Pairs around Massive Stars? , 2019, The Astronomical Journal.

[23]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[24]  Jeffrey K. Uhlmann,et al.  Unscented filtering and nonlinear estimation , 2004, Proceedings of the IEEE.

[25]  Luca Martino,et al.  Effective sample size for importance sampling based on discrepancy measures , 2016, Signal Process..

[26]  Andrew J Majda,et al.  Quantifying uncertainty in climate change science through empirical information theory , 2010, Proceedings of the National Academy of Sciences.

[27]  Brandon M. Turner,et al.  Journal of Mathematical Psychology a Tutorial on Approximate Bayesian Computation , 2022 .

[28]  V. Elvira,et al.  Compressed Monte Carlo for Distributed Bayesian Inference , 2018 .

[29]  Reinhard Genzel,et al.  An Update on Monitoring Stellar Orbits in the Galactic Center , 2016, 1611.09144.

[30]  Neus Sabater,et al.  Emulation of Leaf, Canopy and Atmosphere Radiative Transfer Models for Fast Global Sensitivity Analysis , 2016, Remote. Sens..

[31]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[32]  Petar M. Djuric,et al.  Gaussian sum particle filtering , 2003, IEEE Trans. Signal Process..

[33]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[34]  Philip C. Gregory,et al.  Bayesian re-analysis of the Gliese 581 exoplanet system , 2011, 1101.0800.

[35]  Petar M. Djuric,et al.  Gaussian particle filtering , 2003, IEEE Trans. Signal Process..

[36]  Peter C. Young,et al.  Statistical Emulation of Large Linear Dynamic Models , 2011, Technometrics.

[37]  Slawomir Koziel,et al.  Multi-Level CFD-Based Airfoil Shape Optimization With Automated Low-Fidelity Model Selection , 2013, ICCS.

[38]  Luca Martino,et al.  Group Importance Sampling for Particle Filtering and MCMC , 2017, Digit. Signal Process..

[39]  David Duvenaud,et al.  Optimally-Weighted Herding is Bayesian Quadrature , 2012, UAI.

[40]  R. Rebolo,et al.  HADES RV program with HARPS-N at the TNG , 2019, Astronomy & Astrophysics.

[41]  A. OHagan,et al.  Bayesian analysis of computer code outputs: A tutorial , 2006, Reliab. Eng. Syst. Saf..

[42]  Luca Martino,et al.  Active emulation of computer codes with Gaussian processes - Application to remote sensing , 2019, Pattern Recognit..

[43]  W. Verhoef,et al.  PROSPECT+SAIL models: A review of use for vegetation characterization , 2009 .

[44]  Slawomir Koziel,et al.  Physics-Based Surrogates for Low-Cost Modeling of Microwave Structures , 2013, ICCS.

[45]  I. Plauchu-Frayn,et al.  Discovery of WASP-113b and WASP-114b, two inflated hot-Jupiters with contrasting densities , 2016, 1607.02341.

[46]  Neil D. Lawrence,et al.  Deep Gaussian Processes for Multi-fidelity Modeling , 2019, ArXiv.

[47]  Luca Martino,et al.  Weighting a resampled particle in Sequential Monte Carlo , 2016, 2016 IEEE Statistical Signal Processing Workshop (SSP).

[48]  Petar M. Djuric,et al.  Resampling Algorithms for Particle Filters: A Computational Complexity Perspective , 2004, EURASIP J. Adv. Signal Process..

[49]  Pierre L'Ecuyer,et al.  Efficiency improvement and variance reduction , 1994, Proceedings of Winter Simulation Conference.

[50]  F. Fekel,et al.  A catalog of chromospherically active binary stars (second edition) , 1993 .

[51]  Tiancheng Li,et al.  Deterministic resampling: Unbiased sampling to avoid sample impoverishment in particle filters , 2012, Signal Process..