Architectures for efficient implementation of particle filters

Particle filters are sequential Monte Carlo methods that are used in numerous problems where time-varying signals must be presented in real time and where the objective is to estimate various unknowns of the signal and/or detect events described by the signals. The standard solutions of such problems in many applications are based on the Kalman filters or extended Kalman filters. In situations when the problems are nonlinear or the noise that distorts the signals is non-Gaussian, the Kalman filters provide a solution that may be far from optimal. Particle filters are an intriguing alternative to the Kalman filters due to their excellent performance in very difficult problems including communications, signal processing, navigation, and computer vision. Hence, particle filters have been the focus of wide research recently and immense literature can be found on their theory. Most of these works recognize the complexity and computational intensity of these filters, but there has been no effort directed toward the implementation of these filters in hardware. The objective of this dissertation is to develop, design, and build efficient hardware for particle filters, and thereby bring them closer to practical applications. The fact that particle filters outperform most of the traditional filtering methods in many complex practical scenarios, coupled with the challenges related to decreasing their computational complexity and improving real-time performance, makes this work worthwhile. The main goals of this dissertation are to develop and modify particle filter algorithms and to develop physically feasible hardware architectures that allow for improving the processing speed of particle filters. The issues tackled include reduction of computational complexity, improving scalability of parallel implementation and reducing memory requirements. This work has resulted in the development of the first hardware prototype of a particle filter. The speed improvement in comparison with the implementation on the state-of-the-art digital signal processors is fifty times.

[1]  Harold W. Sorenson,et al.  Recursive Bayesian estimation using piece-wise constant approximations , 1988, Autom..

[2]  R. S. Bucy,et al.  Bayes Theorem and Digital Realizations for Non-Linear Filters , 1969 .

[3]  Ronald L. Wasserstein,et al.  Monte Carlo: Concepts, Algorithms, and Applications , 1997 .

[4]  Seehyun Kim,et al.  Fixed-point optimization utility for C and C++ based digital signal processing programs , 1998 .

[5]  David A. Patterson,et al.  Computer Architecture - A Quantitative Approach, 5th Edition , 1996 .

[6]  Petar M. Djuric,et al.  Gaussian particle filtering , 2003, IEEE Trans. Signal Process..

[7]  Richard J. Meinhold,et al.  Robustification of Kalman Filter Models , 1989 .

[8]  W. Gilks,et al.  Following a moving target—Monte Carlo inference for dynamic Bayesian models , 2001 .

[9]  L. Devroye.  Non-Uniform Random Variate Generation , 1986 .

[10]  Jun S. Liu,et al.  Sequential Monte Carlo methods for dynamic systems , 1997 .

[11]  M. West,et al.  Dynamic Generalized Linear Models and Bayesian Forecasting , 1985 .

[12]  Scott Hauck,et al.  Reconfigurable computing: a survey of systems and software , 2002, CSUR.

[13]  Petar M. Djuric,et al.  Resampling algorithms and architectures for distributed particle filters , 2005, IEEE Transactions on Signal Processing.

[14]  Andreas Gerstlauer,et al.  System Design - A Practical Guide with SpecC , 2001 .

[15]  Neil J. Gordon,et al.  Editors: Sequential Monte Carlo Methods in Practice , 2001 .

[16]  Xiaodong Wang,et al.  Monte Carlo filters for adaptive detection in fading channels , 1999, Conference Record of the Thirty-Third Asilomar Conference on Signals, Systems, and Computers (Cat. No.CH37020).

[17]  Sujit Dey,et al.  High-Level Power Analysis and Optimization , 1997 .

[18]  Samuel T. Chanson,et al.  Process groups and group communications: classifications and requirements , 1990, Computer.

[19]  Sangjin Hong,et al.  Architectures and memory schemes for sampling and resampling in particle filters , 2004, 3rd IEEE Signal Processing Education Workshop. 2004 IEEE 11th Digital Signal Processing Workshop, 2004..

[20]  N. Oudjane,et al.  Progressive correction for regularized particle filters , 2000, Proceedings of the Third International Conference on Information Fusion.

[21]  Jun S. Liu,et al.  Sequential Imputations and Bayesian Missing Data Problems , 1994 .

[22]  Petar M. Djurić.  Sequential Estimation of Signals under Model Uncertainty , 2001, Sequential Monte Carlo Methods in Practice.

[23]  H. Sorenson,et al.  Nonlinear Bayesian estimation using Gaussian sum approximations , 1972 .

[24]  M. West,et al.  Bayesian Dynamic Factor Models and Portfolio Allocation , 2000 .

[25]  Nicholas G. Polson,et al.  A Monte Carlo Approach to Nonnormal and Nonlinear State-Space Modeling , 1992 .

[26]  Andrew Blake,et al.  A Probabilistic Exclusion Principle for Tracking Multiple Objects , 2004, International Journal of Computer Vision.

[27]  Jean-Luc Danger,et al.  Efficient FPGA implementation of Gaussian noise generator for communication channel emulation , 2000, ICECS 2000. 7th IEEE International Conference on Electronics, Circuits and Systems (Cat. No.00EX445).

[28]  Jack E. Volder.  The CORDIC Trigonometric Computing Technique , 1959, IRE Trans. Electron. Comput..

[29]  Viktor K. Prasanna,et al.  A model-based methodology for application specific energy efficient data path design using FPGAs , 2002, Proceedings IEEE International Conference on Application- Specific Systems, Architectures, and Processors.

[30]  Stuart J. Russell,et al.  Stochastic simulation algorithms for dynamic probabilistic networks , 1995, UAI.

[31]  Mark A. Richards,et al.  Chips, Architectures and Algorithms: Reflections on the Exponential Growth of Digital Signal Processing Capability , 2003 .

[32]  P. Djurić,et al.  Particle filtering , 2003, IEEE Signal Process. Mag..

[33]  Niclas Bergman,et al.  Recursive Bayesian Estimation : Navigation and Tracking Applications , 1999 .

[34]  Sangjin Hong,et al.  Finite precision effect on performance and complexity of particle filters for bearing-only tracking , 2002, Conference Record of the Thirty-Sixth Asilomar Conference on Signals, Systems and Computers, 2002..

[35]  Hugh F. Durrant-Whyte,et al.  A new method for the nonlinear transformation of means and covariances in filters and estimators , 2000, IEEE Trans. Autom. Control..

[36]  Christian Musso,et al.  Improving Regularised Particle Filters , 2001, Sequential Monte Carlo Methods in Practice.

[37]  A. Winsor.  Sampling techniques. , 2000, Nursing times.

[38]  C. Dou.  Particle Filtering for Demodulation in Fading Channels with Non-Gaussian Additive Noise. , 2001 .

[39]  Michael J. Flynn,et al.  Detection and Parallel Execution of Independent Instructions , 1970, IEEE Transactions on Computers.

[40]  Keshab K. Parhi,et al.  VLSI digital signal processing systems , 1999 .

[41]  Brian D. Ripley,et al.  Stochastic Simulation , 2005 .

[42]  J. Geweke,et al.  Bayesian Inference in Econometric Models Using Monte Carlo Integration , 1989 .

[43]  S. Pasupathy,et al.  Adaptive MLSD receiver with identification of flat fading channels , 1997, 1997 IEEE 47th Vehicular Technology Conference. Technology in Motion.

[44]  Eric P. Fox.  Bayesian Statistics 3 , 1991 .

[45]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[46]  Petar M. Djuric,et al.  A design complexity comparison method for loop-based signal processing algorithms: particle filters , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[47]  Jan M. Rabaey,et al.  VLSI design and implementation fuels the signal-processing revolution , 1998 .

[48]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[49]  Simon J. Godsill,et al.  Improvement Strategies for Monte Carlo Particle Filters , 2001, Sequential Monte Carlo Methods in Practice.

[50]  P. Fearnhead,et al.  An improved particle filter for non-linear problems , 1999 .

[51]  Simon J. Godsill,et al.  Monte Carlo filtering and smoothing with application to time-varying spectral estimation , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[52]  Jun S. Liu,et al.  Rejection Control and Sequential Importance Sampling , 1998 .

[53]  Nando de Freitas,et al.  The Unscented Particle Filter , 2000, NIPS.

[54]  David A. Patterson,et al.  Computer Architecture: A Quantitative Approach , 1969 .

[55]  J. Geweke,et al.  Antithetic acceleration of Monte Carlo integration in Bayesian inference , 1988 .

[56]  Masahito Yamada,et al.  Structural Time Series Models and the Kalman Filter , 1989 .

[57]  Anoop Gupta,et al.  Parallel computer architecture - a hardware / software approach , 1998 .

[58]  N. G. Best,et al.  Dynamic conditional independence models and Markov chain Monte Carlo methods , 1997 .

[59]  Kazufumi Ito,et al.  Gaussian filters for nonlinear filtering problems , 2000, IEEE Trans. Autom. Control..

[60]  C. Masreliez.  Approximate non-Gaussian filtering with linear state and observation relations , 1975 .

[61]  David L. Kuck,et al.  The Structure of Computers and Computations , 1978 .

[62]  Norman P. Jouppi,et al.  Available instruction-level parallelism for superscalar and superpipelined machines , 1989, ASPLOS III.

[63]  Earl E. Swartzlander,et al.  A unified view of CORDIC processor design , 1996 .

[64]  Petar M. Djuric,et al.  Resampling Algorithms for Particle Filters: A Computational Complexity Perspective , 2004, EURASIP J. Adv. Signal Process..

[65]  Bruno Haller,et al.  Dedicated VLSI Architectures for Adaptive Interference Suppression in Wireless Communication Systems , 2002 .

[66]  H. Sorenson,et al.  Recursive bayesian estimation using gaussian sums , 1971 .

[67]  M. Pitt,et al.  Filtering via Simulation: Auxiliary Particle Filters , 1999 .

[68]  Rong Chen,et al.  Adaptive joint detection and decoding in flat-fading channels via mixture Kalman filtering , 2000, IEEE Trans. Inf. Theory.

[69]  Cyril Fonlupt,et al.  Analysis of Synchronous Dynamic Load Balancing Algorithms , 1995, PARCO.

[70]  Jan M. Rabaey,et al.  Behavioral level guidance using property-based design characterization , 1996 .

[71]  Petar M. Djuric,et al.  Algorithmic modification of Particle Filters for hardware implementation , 2004, 2004 12th European Signal Processing Conference.

[72]  Petar M. Djuric,et al.  New resampling algorithms for particle filters , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[73]  A.C.J. Kienhuis,et al.  Design Space Exploration of Stream-based Dataflow Architectures , 1999 .

[74]  B. Anderson,et al.  Optimal Filtering , 1979, IEEE Transactions on Systems, Man, and Cybernetics.

[75]  Russ Miller,et al.  Meshes with reconfigurable buses , 1988 .

[76]  Jun S. Liu,et al.  Blind Deconvolution via Sequential Imputations , 1995 .

[77]  Martin C. Herbordt,et al.  Message-passing algorithms for a SIMD torus with coteries , 1991, SPAA '90.

[78]  Sangjin Hong,et al.  Performance and complexity analysis of adaptive particle filtering for tracking applications , 2002, Conference Record of the Thirty-Sixth Asilomar Conference on Signals, Systems and Computers, 2002..

[79]  R. Kohn,et al.  On Gibbs sampling for state space models , 1994 .

[80]  Simon J. Godsill,et al.  On sequential Monte Carlo sampling methods for Bayesian filtering , 2000, Stat. Comput..

[81]  S. Frühwirth-Schnatter.  Applied state space modelling of non-Gaussian time series using integration-based Kalman filtering , 1994 .

[82]  Petar M. Djuric,et al.  Design and Implementation of Flexible Resampling Mechanism for High-Speed Parallel Particle Filters , 2006, J. VLSI Signal Process..

[83]  Peter Pirsch,et al.  Architectures for digital signal processing , 1998 .

[84]  Nelson L. Passos.  Pipelined And Parallel Computer Architectures , 1998, IEEE Concurrency.

[85]  Petar M. Djuric,et al.  An efficient fixed-point implementation of residual resampling scheme for high-speed particle filters , 2004, IEEE Signal Processing Letters.

[86]  Alexandru Nicolau,et al.  Measuring the Parallelism Available for Very Long Instruction Word Architectures , 1984, IEEE Transactions on Computers.

[87]  Michael Wolfe,et al.  High performance compilers for parallel computing , 1995 .

[88]  T. Kloek,et al.  Bayesian estimates of equation system parameters, An application of integration by Monte Carlo , 1976 .

[89]  Nando de Freitas,et al.  Sequential Monte Carlo Methods in Practice , 2001, Statistics for Engineering and Information Science.

[90]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[91]  R. E. Kalman,et al.  New Results in Linear Filtering and Prediction Theory , 1961 .

[92]  Simon J. Godsill,et al.  Fixed-lag smoothing using sequential importance sampling , 1999 .

[93]  Manoj Kumar,et al.  Measuring Parallelism in Computation-Intensive Scientific/Engineering Applications , 1988, IEEE Trans. Computers.

[94]  P. Kumar,et al.  Theory and practice of recursive identification , 1985, IEEE Transactions on Automatic Control.

[95]  J. Huang,et al.  Curse of dimensionality and particle filters , 2003, 2003 IEEE Aerospace Conference Proceedings (Cat. No.03TH8652).

[96]  Ralph Duncan.  A survey of parallel computer architectures , 1990, Computer.

[97]  R. Kohn,et al.  Markov chain Monte Carlo in conditionally Gaussian state space models , 1996 .

[98]  Nicholas G. Polson,et al.  Particle Filtering , 2006 .

[99]  Andrew Harvey,et al.  Forecasting, Structural Time Series Models and the Kalman Filter , 1990 .

[100]  Peter J. W. Rayner,et al.  Digital Audio Restoration: A Statistical Model Based Approach , 1998 .