High-level primitives for recursive maximum likelihood estimation

This paper proposes a high-level language constituted of a small number of primitives and macros for describing recursive maximum likelihood (ML) estimation algorithms. This language is applicable to estimation problems involving linear Gaussian models or processes taking values in a finite set. The use of high-level primitives allows the development of highly modular ML estimation algorithms based on simple numerical building blocks. The primitives, which correspond to the combination of different measurements, the extraction of sufficient statistics, and the conversion of the status of a variable from unknown to observed, or vice versa, are first defined for linear Gaussian relations specifying mixed deterministic/stochastic information about the system variables. These primitives are used to define other macros and are illustrated by deriving new filtering and smoothing algorithms for linear descriptor systems. The primitives are then extended to finite state processes and used to implement the Viterbi ML state sequence estimator for a hidden Markov model.

[1]  E. L. Lehmann,et al.  Theory of point estimation , 1950 .

[2]  C. Striebel,et al.  On the maximum likelihood estimates for linear dynamic systems , 1965 .

[3]  David Q. Mayne,et al.  A solution of the smoothing problem for linear dynamic systems , 1966, Autom..

[4]  Andrew J. Viterbi,et al.  Error bounds for convolutional codes and an asymptotically optimum decoding algorithm , 1967, IEEE Trans. Inf. Theory.

[5]  D. C. Fraser,et al.  A new technique for the optimal smoothing of data , 1968 .

[6]  Frank Harary,et al.  Graph Theory , 2016 .

[7]  G. David Forney,et al.  Maximum-likelihood sequence estimation of digital sequences in the presence of intersymbol interference , 1972, IEEE Trans. Inf. Theory.

[8]  Jr. G. Forney,et al.  The viterbi algorithm , 1973 .

[9]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[10]  Alan S. Willsky,et al.  A survey of design methods for failure detection in dynamic systems , 1976, Autom..

[11]  G. Bierman.  Factorization methods for discrete sequential estimation , 1977 .

[12]  J. Laurie Snell,et al.  Markov Random Fields and Their Applications , 1980 .

[13]  V.W.S. Chan,et al.  Principles of Digital Communication and Coding , 1979 .

[14]  B. Anderson,et al.  Optimal Filtering , 1979, IEEE Transactions on Systems, Man, and Cybernetics.

[15]  A. Willsky,et al.  Analytical redundancy and the design of robust failure detection systems , 1984 .

[16]  H. W. Sorenson,et al.  Kalman filtering : theory and application , 1985 .

[17]  L. Rabiner,et al.  An introduction to hidden Markov models , 1986, IEEE ASSP Magazine.

[18]  Judea Pearl,et al.  Fusion, Propagation, and Structuring in Belief Networks , 1986, Artif. Intell..

[19]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[20]  Anil K. Jain,et al.  Random field models in image analysis , 1989 .

[21]  Filtrage et lissage des systèmes implicites discrets , 1989 .

[22]  J. Fort,et al.  Stochastic Processes on a Lattice and Gibbs Measures , 1990 .

[23]  Ross D. Shachter,et al.  Fusion and Propagation with Multiple Observations in Belief Networks , 1991, Artif. Intell..

[24]  Thierry Gautier,et al.  Programming real-time applications with SIGNAL , 1991, Proc. IEEE.

[25]  A. Benveniste.  Constructive probability and the SIGNalea language: building and handling random processes via programming , 1992, [1992] Proceedings of the 31st IEEE Conference on Decision and Control.

[26]  Ramine Nikoukhah,et al.  Graph structure and recursive estimation of noisy linear relations , 1992 .

[27]  A. Willsky,et al.  Kalman filtering and Riccati equations for descriptor systems , 1992 .

[28]  Luigi Chisci,et al.  Square-root Kalman filtering of descriptor systems , 1992 .

[29]  Michèle Basseville,et al.  On the Use of Descriptor Systems for Failure Detection and Isolation , 1993 .

[30]  Ali H. Sayed,et al.  Recursive linear estimation in Krein spaces. I. Theory , 1993, Proceedings of 32nd IEEE Conference on Decision and Control.

[31]  Bernard Delyon.  Remarks on filtering of semi-markov data , 1993 .

[32]  Albert Benveniste,et al.  A Calculus of Stochastic Systems : Specification, Simulation, and Hidden State Estimation , 1994 .

[33]  J. Paris.  The Uncertain Reasoner's Companion: A Mathematical Perspective , 1994 .

[34]  John B. Moore,et al.  Hidden Markov Models: Estimation and Control , 1994 .

[35]  Ramine Nikoukhah,et al.  Innovations generation in the presence of unknown inputs: Application to robust failure detection , 1994, Autom..

[36]  Albert Benveniste,et al.  A Calculus of Stochastic Systems for the Specification, Simulation, and Hidden State Estimation of Mixed Stochastic/Nonstochastic Systems , 1994, Theor. Comput. Sci..

[37]  Michèle Basseville,et al.  Detection of Abrupt Changes: Theory and Applications. , 1995 .

[38]  Bernard Delyon,et al.  Remarks on linear and nonlinear filtering , 1995, IEEE Trans. Inf. Theory.

[39]  Albert Benveniste,et al.  Multiscale Signal Processing: From QMF to Wavelets , 1995 .

[40]  T. Kailath,et al.  Linear estimation in Krein spaces. I. Theory , 1996, IEEE Trans. Autom. Control..

[41]  A. Bonato,et al.  Graphs and Hypergraphs , 2022 .