Seven dubious methods to mitigate stiffness in particle flow with non-zero diffusion for nonlinear filters, Bayesian decisions, and transport

We explain why one of our favorite particle flows (method #18) requires special care for numerical integration. The root cause is stiffness of the flow. This can be mitigated with a small increase in computational complexity (a factor of three). There are many ways to do this, but some are much better than others. In particular, one can use a smaller step size for numerical integration, or use principal coordinates, or use a stiff ODE solver or an adaptive ODE solver or other methods. We show that the best methods are contrary to the standard advice given in textbooks.

[1]  Tim B. Swartz,et al.  Approximating Integrals Via Monte Carlo and Deterministic Methods , 2000 .

[2]  C. Villani.  Topics in Optimal Transportation , 2003 .

[3]  Robert D. Russell,et al.  Optimal mass transport for higher dimensional adaptive grid generation , 2011, J. Comput. Phys..

[4]  A. Shnirelman,et al.  Evolution of singularities, generalized Liapunov function and generalized integral for an ideal incompressible fluid , 1997 .

[5]  Fred Daum,et al.  Particle degeneracy: root cause and solution , 2011, Defense + Commercial Sensing.

[6]  Omar Hijab.  A class of infinite dimensional filters , 1980, 1980 19th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[7]  Fred Daum,et al.  Hollywood log-homotopy: movies of particle flow for nonlinear filters , 2011, Defense + Commercial Sensing.

[8]  Fred Daum,et al.  Friendly rebuttal to Chen and Mehra: incompressible particle flow for nonlinear filters , 2012, Defense + Commercial Sensing.

[9]  R. Fitzgerald,et al.  Decoupled Kalman filters for phased array radar tracking , 1983 .

[10]  Fred Daum,et al.  Particle flow inspired by Knothe-Rosenblatt transport for nonlinear filters , 2013, Defense, Security, and Sensing.

[11]  Fred Daum,et al.  Nonlinear filters with log-homotopy , 2007, SPIE Optical Engineering + Applications.

[12]  Gian Luca Delzanno,et al.  The fluid dynamic approach to equidistribution methods for grid generation and adaptation , 2009 .

[13]  M. V. Kulikova,et al.  Accurate Numerical Implementation of the Continuous-Discrete Extended Kalman Filter , 2014, IEEE Transactions on Automatic Control.

[14]  Frederick Daum.  A new nonlinear filtering formula non-Gaussian discrete time measurements , 1986, 1986 25th IEEE Conference on Decision and Control.

[15]  Xiao-Li Meng,et al.  Simulating Normalizing Constants: From Importance Sampling to Bridge Sampling to Path Sampling , 1998 .

[16]  Peter K. Jimack,et al.  Moving mesh methods for solving parabolic partial differential equations , 2011 .

[17]  Fred Daum,et al.  Seventeen dubious methods to approximate the gradient for nonlinear filters with particle flow , 2009, Optical Engineering + Applications.

[18]  Fred Daum,et al.  Exact particle flow for nonlinear filters , 2010, Defense + Commercial Sensing.

[19]  Lawrence F. Shampine,et al.  Ill-conditioned matrices and the integration of stiff ODEs , 1993 .

[20]  Fred Daum,et al.  Coulomb's law particle flow for nonlinear filters , 2011, Optical Engineering + Applications.

[21]  Gian Luca Delzanno,et al.  The fluid dynamic approach to equidistribution methods for grid adaptation , 2011, Comput. Phys. Commun..

[22]  Mark R. Morelande,et al.  Optimal parameterization of posterior densities using homotopy , 2011, 14th International Conference on Information Fusion.

[23]  Erich Novak,et al.  Simple Monte Carlo and the Metropolis algorithm , 2007, J. Complex..

[24]  Christian Musso,et al.  Improving Regularised Particle Filters , 2001, Sequential Monte Carlo Methods in Practice.

[25]  Fred Daum,et al.  Fourier transform particle flow for nonlinear filters , 2013, Defense, Security, and Sensing.

[26]  Fred Daum,et al.  Numerical experiments for nonlinear filters with exact particle flow induced by log-homotopy , 2010, Defense + Commercial Sensing.

[27]  J. Lions.  Optimal Control of Systems Governed by Partial Differential Equations , 1971 .

[28]  Allen R. Tannenbaum,et al.  An Efficient Numerical Method for the Solution of the L2 Optimal Mass Transfer Problem , 2010, SIAM J. Sci. Comput..

[29]  A. Doucet,et al.  A Tutorial on Particle Filtering and Smoothing: Fifteen years later , 2008 .

[30]  M. Girolami,et al.  Riemann manifold Langevin and Hamiltonian Monte Carlo methods , 2011, Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[31]  Fred Daum,et al.  Generalized particle flow for nonlinear filters , 2010, Defense + Commercial Sensing.

[32]  Ron Sacks-Davis,et al.  A review of recent developments in solving ODEs , 1985, CSUR.

[33]  L. Trefethen,et al.  Stiffness of ODEs , 1993 .

[34]  Uwe D. Hanebeck,et al.  Progressive Bayes: a new framework for nonlinear state estimation , 2003, SPIE Defense + Commercial Sensing.

[35]  Fred Daum,et al.  Zero curvature particle flow for nonlinear filters , 2012, Defense, Security, and Sensing.

[36]  Fred Daum,et al.  Particle flow with non-zero diffusion for nonlinear filters , 2013, Defense, Security, and Sensing.

[37]  Claes Johnson,et al.  Explicit Time-Stepping for Stiff ODEs , 2003, SIAM J. Sci. Comput..

[38]  D. Rudolf,et al.  Hit-and-Run for Numerical Integration , 2012, 1212.4486.

[39]  H. Rue,et al.  Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations , 2009 .

[40]  Frederick E. Daum,et al.  Particle flow and Monge-Kantorovich transport , 2012, 2012 15th International Conference on Information Fusion.

[41]  Fred Daum,et al.  A fresh perspective on research for nonlinear filters , 2010, Defense + Commercial Sensing.

[42]  Fred Daum,et al.  Exact particle flow for nonlinear filters: Seventeen dubious solutions to a first order linear underdetermined PDE , 2010, 2010 Conference Record of the Forty Fourth Asilomar Conference on Signals, Systems and Computers.

[43]  Daniel Rudolf,et al.  Explicit error bounds for lazy reversible Markov chain Monte Carlo , 2008, J. Complex..

[44]  R. Chartrand,et al.  A Gradient Descent Solution to the Monge-Kantorovich Problem , 2009 .

[45]  Branko Ristic,et al.  Beyond the Kalman Filter: Particle Filters for Tracking Applications , 2004 .

[46]  F. Daum.  Nonlinear filters: beyond the Kalman filter , 2005, IEEE Aerospace and Electronic Systems Magazine.

[47]  Frederick E. Daum.  Exact Finite Dimensional Filters for Cryptodeterministic Systems , 1986, 1986 American Control Conference.

[48]  Lingji Chen,et al.  A study of nonlinear filters with particle flow induced by log-homotopy , 2010, Defense + Commercial Sensing.

[49]  Robert D. Russell,et al.  Adaptivity with moving grids , 2009, Acta Numerica.

[50]  Simon J. Godsill,et al.  Improvement Strategies for Monte Carlo Particle Filters , 2001, Sequential Monte Carlo Methods in Practice.

[51]  Fred Daum,et al.  Numerical experiments for Coulomb's law particle flow for nonlinear filters , 2011, Optical Engineering + Applications.

[52]  C. Villani.  The founding fathers of optimal transport , 2009 .

[53]  Frederick Daum.  A new nonlinear filtering formula for discrete time measurements , 1985, 1985 24th IEEE Conference on Decision and Control.

[54]  Fred Daum,et al.  Small curvature particle flow for nonlinear filters , 2012, Defense + Commercial Sensing.