Discrete-time, discrete-valued observable operator models: a tutorial

This tutorial gives a basic yet rigorous introduction to observable operator models (OOMs). OOMs are a recently discovered class of models of stochastic processes. They are mathematically simple in that they require only concepts from elementary linear algebra. The linear algebra nature gives rise to an efficient, consistent, unbiased, constructive learning procedure for estimating models from empirical data. The tutorial describes in detail the mathematical foundations and the practical use of OOMs for identifying and predicting discrete-time, discrete-valued processes, both for output-only and input-output systems. key words: stochastic time series, system identification, observable operator models

[1]  J. Doob.  Stochastic processes , 1953 .

[2]  D. Blackwell,et al.  On the Identifiability Problem for Functions of Finite Markov Chains , 1957 .

[3]  J. Davenport.  Editor , 1960 .

[4]  A. Heller.  On Stochastic Processes Derived From Markov Chains , 1965 .

[5]  Radu Theodorescu,et al.  Random processes and learning , 1969 .

[6]  Edward J. Sondik,et al.  The Optimal Control of Partially Observable Markov Processes over a Finite Horizon , 1973, Oper. Res..

[7]  I. Gihman,et al.  Controlled Stochastic Processes , 1979 .

[8]  Editors , 1986, Brain Research Bulletin.

[9]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[10]  Thomas G. Dietterich,et al.  In Advances in Neural Information Processing Systems 12 , 1991, NIPS 1991.

[11]  Shun-ichi Amari,et al.  Identifiability of hidden Markov information sources and their minimum degrees of freedom , 1992, IEEE Trans. Inf. Theory.

[12]  Robert J. Plemmons,et al.  Nonnegative Matrices in the Mathematical Sciences , 1979, Classics in Applied Mathematics.

[13]  John B. Moore,et al.  Hidden Markov Models: Estimation and Control , 1994 .

[14]  Gene H. Golub,et al.  Matrix Computations, Third Edition , 1996 .

[15]  H. Jaeger.  A short introduction to observable operator models of discrete stochastic processes ∗ , 1997 .

[16]  H. Jaeger,et al.  Observable operator models II: Interpretable models and model induction , 1997 .

[17]  Kumpati S. Narendra,et al.  Identification and control , 1998 .

[18]  H. Jaeger.  Action selection for stochastic, delayed reward ? , 1999 .

[19]  Sadri Hassani,et al.  Nonlinear Dynamics and Chaos , 2000 .

[20]  Herbert Jaeger,et al.  Observable Operator Models for Discrete Stochastic Time Series , 2000, Neural Computation.

[21]  Peter Stone,et al.  Learning Predictive State Representations , 2003, ICML.

[22]  Yoshua Bengio,et al.  Markovian Models for Sequential Data , 2004 .