Renormalization group flow in k-space for nonlinear filters, Bayesian decisions and transport

We derive a new algorithm which avoids normalization of the probability density for particle flow. The algorithm was inspired by renormalization group flow in quantum field theory. In contrast with other particle flow algorithms, this one works in k-space rather than state space. We have roughly 30 or 40 algorithms to compute particle flow, and the three best algorithms avoid computing the normalization of the conditional probability density of the state. We explain why explicit normalization often spoils the flow. This phenomenon has been noticed by other researchers for completely different applications (e.g., weather prediction), but apparently the benefits of avoiding normalization are not well known.

[1]  Daniel Rudolf,et al.  Explicit error bounds for lazy reversible Markov chain Monte Carlo , 2008, J. Complex..

[2]  D. Rudolf,et al.  Hit-and-Run for Numerical Integration , 2012, 1212.4486.

[3]  R. Chartrand,et al.  A Gradient Descent Solution to the Monge-Kantorovich Problem , 2009 .

[4]  Fred Daum,et al.  Renormalization group flow and other ideas inspired by physics for nonlinear filters, Bayesian decisions, and transport , 2014, Defense + Security Symposium.

[5]  Frederick Daum.  A new nonlinear filtering formula for discrete time measurements , 1985, 1985 24th IEEE Conference on Decision and Control.

[6]  Fred Daum,et al.  Small curvature particle flow for nonlinear filters , 2012, Defense + Commercial Sensing.

[7]  Fred Daum,et al.  Friendly rebuttal to Chen and Mehra: incompressible particle flow for nonlinear filters , 2012, Defense + Commercial Sensing.

[8]  Allen R. Tannenbaum,et al.  An Efficient Numerical Method for the Solution of the L2 Optimal Mass Transfer Problem , 2010, SIAM J. Sci. Comput..

[9]  H. Rue,et al.  Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations , 2009 .

[10]  J. Stoyanov.  The Oxford Handbook of Nonlinear Filtering , 2012 .

[11]  A. Doucet,et al.  A Tutorial on Particle Filtering and Smoothing: Fifteen years later , 2008 .

[12]  Y. Brenier,et al.  RECONSTRUCTION OF THE EARLY UNIVERSE, ZELDOVICH APPROXIMATION AND MONGE-AMPÈRE GRAVITATION , 2010 .

[13]  Fred Daum,et al.  Zero curvature particle flow for nonlinear filters , 2012, Defense, Security, and Sensing.

[14]  C. Villani.  Topics in Optimal Transportation , 2003 .

[15]  Rien van de Weygaert,et al.  The Zel'dovich approximation: key to understanding cosmic web complexity , 2013, 1311.7134.

[16]  Christian Musso,et al.  Improving Regularised Particle Filters , 2001, Sequential Monte Carlo Methods in Practice.

[17]  Tim B. Swartz,et al.  Approximating Integrals Via Monte Carlo and Deterministic Methods , 2000 .

[18]  Erich Novak,et al.  Simple Monte Carlo and the Metropolis algorithm , 2007, J. Complex..

[19]  Fred Daum,et al.  Numerical experiments for nonlinear filters with exact particle flow induced by log-homotopy , 2010, Defense + Commercial Sensing.

[20]  M. White.  The Zel'dovich approximation , 2014, 1401.5466.

[21]  Nando de Freitas,et al.  Sequential Monte Carlo Methods in Practice , 2001, Statistics for Engineering and Information Science.

[22]  Fred Daum,et al.  Particle flow with non-zero diffusion for nonlinear filters , 2013, Defense, Security, and Sensing.

[23]  Robert D. Russell,et al.  Optimal mass transport for higher dimensional adaptive grid generation , 2011, J. Comput. Phys..

[24]  A. Shnirelman,et al.  Evolution of singularities, generalized Liapunov function and generalized integral for an ideal incompressible fluid , 1997 .

[25]  V. Arnold,et al.  Topological methods in hydrodynamics , 1998 .

[26]  Frederick E. Daum,et al.  Particle flow and Monge-Kantorovich transport , 2012, 2012 15th International Conference on Information Fusion.

[27]  Fred Daum,et al.  A fresh perspective on research for nonlinear filters , 2010, Defense + Commercial Sensing.

[28]  Peter K. Jimack,et al.  Moving mesh methods for solving parabolic partial differential equations , 2011 .

[29]  Fred Daum,et al.  Particle flow with non-zero diffusion for nonlinear filters, Bayesian decisions and transport , 2013, Optics & Photonics - Optical Engineering + Applications.

[30]  Fred Daum,et al.  Seventeen dubious methods to approximate the gradient for nonlinear filters with particle flow , 2009, Optical Engineering + Applications.

[31]  Fred Daum,et al.  Exact particle flow for nonlinear filters , 2010, Defense + Commercial Sensing.

[32]  Fred Daum,et al.  Hollywood log-homotopy: movies of particle flow for nonlinear filters , 2011, Defense + Commercial Sensing.

[33]  Washek F. Pfeffer,et al.  Distributions for which div v = F has a continuous solution , 2008 .

[34]  Fred Daum,et al.  Proof that particle flow corresponds to Bayes’ rule: necessary and sufficient conditions , 2015, Defense + Security Symposium.

[35]  Uwe D. Hanebeck,et al.  Progressive Bayes: a new framework for nonlinear state estimation , 2003, SPIE Defense + Commercial Sensing.

[36]  Fred Daum,et al.  Seven dubious methods to mitigate stiffness in particle flow with non-zero diffusion for nonlinear filters, Bayesian decisions, and transport , 2014, Defense + Security Symposium.

[37]  Fred Daum,et al.  Fourier transform particle flow for nonlinear filters , 2013, Defense, Security, and Sensing.

[38]  Bernard Dacorogna,et al.  Existence and regularity of solutions of d ! = f with Dirichlet boundary conditions , 2005 .

[39]  Emily Jane Walsh.  Moving mesh methods for problems in meteorology , 2010 .

[40]  Branko Ristic,et al.  Beyond the Kalman Filter: Particle Filters for Tracking Applications , 2004 .

[41]  F. Daum.  Nonlinear filters: beyond the Kalman filter , 2005, IEEE Aerospace and Electronic Systems Magazine.

[42]  Fred Daum,et al.  Particle degeneracy: root cause and solution , 2011, Defense + Commercial Sensing.

[43]  Simon J. Godsill,et al.  Improvement Strategies for Monte Carlo Particle Filters , 2001, Sequential Monte Carlo Methods in Practice.

[44]  J. Lions.  Optimal Control of Systems Governed by Partial Differential Equations , 1971 .

[45]  Fred Daum,et al.  Exact particle flow for nonlinear filters: Seventeen dubious solutions to a first order linear underdetermined PDE , 2010, 2010 Conference Record of the Forty Fourth Asilomar Conference on Signals, Systems and Computers.

[46]  Frederick E. Daum.  Exact Finite Dimensional Filters for Cryptodeterministic Systems , 1986, 1986 American Control Conference.

[47]  Fred Daum,et al.  Numerical experiments for Coulomb's law particle flow for nonlinear filters , 2011, Optical Engineering + Applications.

[48]  Stephen Boyd,et al.  An open letter concerning Subspaces that Minimize the Condition Number of a Matrix , 2009 .

[49]  M. Girolami,et al.  Riemann manifold Langevin and Hamiltonian Monte Carlo methods , 2011, Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[50]  Fred Daum,et al.  A baker’s dozen of new particle flows for nonlinear filters, Bayesian decisions and transport , 2015, Defense + Security Symposium.

[51]  Fred Daum,et al.  Generalized particle flow for nonlinear filters , 2010, Defense + Commercial Sensing.

[52]  N. Gouda,et al.  Why is the zel'dovich approximation so accurate? , 2005 .

[53]  Xiao-Li Meng,et al.  Simulating Normalizing Constants: From Importance Sampling to Bridge Sampling to Path Sampling , 1998 .

[54]  Lingji Chen,et al.  A study of nonlinear filters with particle flow induced by log-homotopy , 2010, Defense + Commercial Sensing.

[55]  Yimin Wei,et al.  Effective condition number and its applications , 2010, Computing.

[56]  Robert D. Russell,et al.  Adaptivity with moving grids , 2009, Acta Numerica.

[57]  Fred Daum,et al.  Coulomb's law particle flow for nonlinear filters , 2011, Optical Engineering + Applications.

[58]  Mark R. Morelande,et al.  Optimal parameterization of posterior densities using homotopy , 2011, 14th International Conference on Information Fusion.

[59]  Fred Daum,et al.  Particle flow inspired by Knothe-Rosenblatt transport for nonlinear filters , 2013, Defense, Security, and Sensing.

[60]  Fred Daum,et al.  Nonlinear filters with log-homotopy , 2007, SPIE Optical Engineering + Applications.

[61]  Gian Luca Delzanno,et al.  The fluid dynamic approach to equidistribution methods for grid generation and adaptation , 2009 .

[62]  C. Villani.  The founding fathers of optimal transport , 2009 .

[63]  Omar Hijab.  A class of infinite dimensional filters , 1980, 1980 19th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[64]  Frederick Daum.  A new nonlinear filtering formula non-Gaussian discrete time measurements , 1986, 1986 25th IEEE Conference on Decision and Control.