A Constrained MDP Approach to Dynamic Quantizer Design for HMM State Estimation

This paper is concerned with dynamic quantizer design for state estimation of hidden Markov models (HMM) using multiple sensors under a sum power constraint at the sensor transmitters. The sensor nodes communicate with a fusion center over temporally correlated flat fading channels modelled by finite state Markov chains. Motivated by energy limitations in sensor nodes, we develop optimal quantizers by minimizing the long term average of the mean square estimation error with a constraint on the long term average of total transmission power across the sensors. Instead of introducing a cost function as a weighted sum of our two objectives, we propose a constrained Markov decision formulation as an average cost problem and employ a linear programming technique to obtain the optimal policy for the constrained problem. Our experimental results assert that the constrained approach is quite efficient in terms of computational complexity and memory requirements for our average cost problem and leads to the same optimal deterministic policies and optimal cost as the unconstrained approach under an irreducibility assumption on the underlying Markov chain and some mild regularity assumptions on the sensor measurement noise processes. We illustrate via numerical studies the performance results for the dynamic quantization scheme. We also study the effect of varying degrees of channel and measurement noise on the performance of the proposed scheme.

[1]  Minyi Huang,et al.  Distributed state estimation for hidden Markov models by sensor networks with dynamic quantization , 2004, Proceedings of the 2004 Intelligent Sensors, Sensor Networks and Information Processing Conference, 2004..

[2]  James C. Spall,et al.  Adaptive stochastic approximation by the simultaneous perturbation method , 2000, IEEE Trans. Autom. Control..

[3]  Subhrakanti Dey,et al.  Dynamic Quantization for Multisensor Estimation Over Bandlimited Fading Channels , 2007, IEEE Transactions on Signal Processing.

[4]  Vijay K. Bhargava,et al.  Optimal and suboptimal packet scheduling over correlated time varying flat fading channels , 2006, IEEE Transactions on Wireless Communications.

[5]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[6]  Subhrakanti Dey,et al.  Information theoretic quantiser design for decentralised estimation of hidden Markov models , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[7]  Subhrakanti Dey,et al.  Dynamic Quantizer Design for Hidden Markov State Estimation Via Multiple Sensors With Fusion Center Feedback , 2006, IEEE Transactions on Signal Processing.

[8]  Venugopal V. Veeravalli,et al.  Decentralized dynamic power control for cellular CDMA systems , 2003, IEEE Trans. Wirel. Commun..

[9]  Vikram Krishnamurthy,et al.  MIMO Transmission Control in Fading Channels—A Constrained Markov Decision Process Formulation With Monotone Randomized Policies , 2007, IEEE Transactions on Signal Processing.

[10]  Tang Liu,et al.  Mobile agent based metadata framework for heterogeneous wireless sensor network , 2010, 2010 International Conference on Educational and Information Technology.

[11]  S. Dey,et al.  Power-efficient dynamic quantization for multisensor HMM state estimation over fading channels , 2008, 2008 3rd International Symposium on Communications, Control and Signal Processing.

[12]  Brian D. O. Anderson,et al.  On state-estimation of a two-state hidden Markov model with quantization , 2001, IEEE Trans. Signal Process..

[13]  Y. Bar-Shalom,et al.  Censoring sensors: a low-communication-rate scheme for distributed detection , 1996, IEEE Transactions on Aerospace and Electronic Systems.

[14]  Lang Tong,et al.  Sensor networks with mobile agents , 2003, IEEE Military Communications Conference, 2003. MILCOM 2003..