Optimal sensor scheduling for Hidden Markov models

Consider the Hidden Markov model where the realization of a single Markov chain is observed by a number of noisy sensors. The sensor scheduling problem for the resulting hidden Markov model is as follows: design an optimal algorithm for selecting at each time instant, one of the many sensors to provide the next measurement. Each measurement has an associated measurement cost. The problem is to select an optimal measurement scheduling policy, so as to minimize a cost function of estimation errors and measurement costs. The problem of determining the optimal measurement policy is solved via stochastic dynamic programming. Numerical results are presented.

[1]  M. Athans.  On the Determination of Optimal Costly Measurement Strategies for Linear Stochastic Systems , 1972 .

[2]  W. Lovejoy.  A survey of algorithmic methods for partially observed Markov decision processes , 1991 .

[3]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[4]  J. Peschon,et al.  Optimal control of measurement subsystems , 1967, IEEE Transactions on Automatic Control.

[5]  A. Bensoussan,et al.  Optimal sensor scheduling in nonlinear filtering of diffusion processes , 1989 .

[6]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[7]  Edward J. Sondik,et al.  The Optimal Control of Partially Observable Markov Processes over a Finite Horizon , 1973, Oper. Res..

[8]  G. Monahan.  State of the Art—A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms , 1982 .

[9]  A. Segall.  Optimal control of noisy finite-state Markov processes , 1977 .

[10]  C. Striebel.  Sufficient statistics in the optimum control of stochastic systems , 1965 .

[11]  Michael Athans,et al.  1972 IFAC congress paper: On the determination of optimal costly measurement strategies for linear stochastic systems , 1972 .

[12]  Douglas Cochran,et al.  Dynamic estimation with selectable linear measurements , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[13]  Edward J. Sondik,et al.  The Optimal Control of Partially Observable Markov Processes over the Infinite Horizon: Discounted Costs , 1978, Oper. Res..

[14]  Karl Johan Åström,et al.  Optimal control of Markov processes with incomplete state information , 1965 .