Nonanticipative Rate Distortion Function and Relations to Filtering Theory

The relation between nonanticipative rate distortion function (RDF) and filtering theory is discussed on abstract spaces. The relation is established by imposing a realizability constraint on the reconstruction conditional distribution of the classical RDF. Existence of the extremum solution of the nonanticipative RDF is shown using weak *-convergence on appropriate topology. The extremum reconstruction conditional distribution is derived in closed form, for the case of stationary processes. The realization of the reconstruction conditional distribution which achieves the infimum of the nonanticipative RDF is described. Finally, an example is presented to illustrate the concepts.

[1]  C. Ionescu Tulcea,et al.  Topics in the Theory of Lifting , 1969 .

[2]  Tong Zhou,et al.  Sensitivity Penalization Based Robust State Estimation for Uncertain Linear Systems , 2010, IEEE Transactions on Automatic Control.

[3]  Richard S. Bucy.  Distortion rate theory and filtering , 1982, IEEE Trans. Inf. Theory.

[4]  J. Galdos,et al.  Information and distortion in reduced-order filter design , 1977, IEEE Trans. Inf. Theory.

[5]  Shunsuke Ihara,et al.  Information theory - for continuous systems , 1993 .

[6]  Sekhar Tatikonda,et al.  Control over noisy channels , 2004, IEEE Transactions on Automatic Control.

[7]  Demosthenis Teneketzis.  Communication in decentralized control , 1980 .

[8]  Alʹbert Nikolaevich Shiri︠a︡ev,et al.  Statistics of random processes , 1977 .

[9]  Serdar Yüksel,et al.  A Tutorial on Quantizer Design for Networked Control Systems : Stabilization and Optimization , 2011 .

[10]  Hugh B. Maynard.  A Radon-Nikodým theorem for finitely additive bounded measures , 1979 .

[11]  Richard M. Murray,et al.  Data Transmission Over Networks for Estimation and Control , 2009, IEEE Transactions on Automatic Control.

[12]  Sean P. Meyn,et al.  Random-Time, State-Dependent Stochastic Drift for Markov Chains and Application to Stochastic Stabilization Over Erasure Channels , 2010, IEEE Transactions on Automatic Control.

[13]  Sekhar Tatikonda,et al.  Markov control problems under communication constraints , 2001, Commun. Inf. Syst..

[14]  Robert M. Gray,et al.  Entropy and Information Theory -2/E. , 2014 .

[15]  Tianyou Chai,et al.  Entropy Optimization Filtering for Fault Isolation of Nonlinear Non-Gaussian Stochastic Systems , 2009, IEEE Transactions on Automatic Control.

[16]  J. Lynch,et al.  A weak convergence approach to the theory of large deviations , 1997 .

[17]  Photios Stavrou,et al.  Causal Rate Distortion Function and Relations to Filtering Theory , 2011, ArXiv.

[18]  Charalambos D. Charalambous,et al.  Stochastic Uncertain Systems Subject to Relative Entropy Constraints: Induced Norms and Monotonicity Properties of Minimax Games , 2007, IEEE Transactions on Automatic Control.

[19]  J. Schwartz,et al.  Linear Operators. Part I: General Theory. , 1960 .

[20]  Charalambos D. Charalambous,et al.  LQG optimality and separation principle for general discrete time partially observed stochastic systems over finite capacity communication channels , 2008, Autom..

[21]  K. Loparo,et al.  Optimal state estimation for stochastic systems: an information theoretic approach , 1997, IEEE Trans. Autom. Control..

[22]  Ina Fourie,et al.  Entropy and Information Theory (2nd ed.) , 2012 .

[23]  Michael Gastpar,et al.  To code, or not to code: lossy source-channel communication revisited , 2003, IEEE Trans. Inf. Theory.

[24]  John B. Moore,et al.  Hidden Markov Models: Estimation and Control , 1994 .

[25]  Charalambos D. Charalambous,et al.  Information Theoretic Bounds for Compound MIMO Gaussian Channels , 2009, IEEE Transactions on Information Theory.

[26]  Ian R. Petersen,et al.  A duality relationship for regular conditional relative entropy , 2005 .

[27]  Photios Stavrou,et al.  Rate Distortion Function for a Class of Relative Entropy Sources , 2013, ArXiv.

[28]  Kevin Barraclough,et al.  I and i , 2001, BMJ : British Medical Journal.

[29]  Richard H. Middleton,et al.  Feedback control performance over a noisy communication channel , 2008, 2008 IEEE Information Theory Workshop.

[30]  Toby Berger,et al.  Rate distortion theory : a mathematical basis for data compression , 1971 .

[31]  P. Graefe.  Linear stochastic systems , 1966 .

[32]  A. Pietsch,et al.  A. I. Tulcea and C. I. Tulcea, Topics in the Theory of Lifting. (Ergebnisse der Mathematik und ihrer Grenzgebiete, Band 48). X + 188 S. Berlin/Heidelberg/New York 1969. Springer‐Verlag. Preis geb. DM 36,‐ . , 1970 .

[33]  T. Kailath,et al.  Indefinite-quadratic estimation and control: a unified approach to H 2 and H ∞ theories , 1999 .

[34]  R. Gray.  Entropy and Information Theory , 1990, Springer New York.

[35]  Charalambos D. Charalambous,et al.  Stability and reliable data reconstruction of uncertain dynamic systems over finite capacity channels , 2010, Autom..

[36]  N. Ahmed,et al.  Rate Distortion Theory for General Sources With Potential Application to Image Compression , 2006 .

[37]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[38]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[39]  A. Shiryayev,et al.  Statistics of Random Processes Ii: Applications , 2000 .

[40]  D. Luenberger.  Optimization by Vector Space Methods , 1968 .