Optimal Kullback-Leibler Aggregation via Spectral Theory of Markov Chains

This paper is concerned with model reduction for complex Markov chain models. The Kullback-Leibler divergence rate is employed as a metric to measure the difference between the Markov model and its approximation. For a certain relaxation of the bi-partition model reduction problem, the solution is shown to be characterized by an associated eigenvalue problem. The form of the eigenvalue problem is closely related to the Markov spectral theory for model reduction. This result is the basis of a heuristic proposed for the m-ary partition problem, resulting in a practical recursive algorithm. The results are illustrated with examples.

[1]  R. Gray.  Entropy and Information Theory , 1990, Springer New York.

[2]  G. Yin,et al.  Discrete-time Markov chains with two-time scales and a countable state space: limit results and queueing applications , 2008 .

[3]  Lihao Xu,et al.  Multiway cuts and spec-tral clustering , 2003 .

[4]  Mathukumalli Vidyasagar,et al.  The 4M (Mixed Memory Markov Model) Algorithm for Finding Genes in Prokaryotic Genomes , 2008, IEEE Transactions on Automatic Control.

[5]  Sean P. Meyn,et al.  On complex spectra and metastability of Markov models , 2008, 2008 47th IEEE Conference on Decision and Control.

[6]  P. Kokotovic,et al.  A singular perturbation approach to modeling and control of Markov chains , 1981 .

[7]  Subhrakanti Dey,et al.  Reduced-complexity filtering for partially observed nearly completely decomposable Markov chains , 2000, IEEE Trans. Signal Process..

[8]  P. Deuflhard,et al.  Robust Perron cluster analysis in conformation dynamics , 2005 .

[9]  O. Junge,et al.  Almost Invariant Sets in Chua's Circuit , 1997 .

[10]  Bruce Hendrickson,et al.  SAND 93-0074 Unlimited Release Printed January 1993 Distribution Category UC-405 Multidimensional Spectral Load Balancing , 2001 .

[11]  Ioannis Kontoyiannis,et al.  Approximating a Diffusion by a Hidden Markov Model , 2009, 0906.0259.

[12]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[13]  A. Bovier,et al.  Metastability in stochastic dynamics of disordered mean-field models , 1998, cond-mat/9811331.

[14]  Eric Vanden-Eijnden,et al.  Optimal Fuzzy Aggregation of Networks , 2010, Multiscale Model. Simul..

[15]  Tiejun Li,et al.  Optimal partition and effective dynamics of complex networks , 2008, Proceedings of the National Academy of Sciences.

[16]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[17]  Prashant G. Mehta,et al.  The Kullback–Leibler Rate Pseudo-Metric for Comparing Dynamical Systems , 2010, IEEE Transactions on Automatic Control.

[18]  P. Deuflhard,et al.  A Direct Approach to Conformational Dynamics Based on Hybrid Monte Carlo , 1999 .

[19]  S. Varadhan,et al.  Asymptotic evaluation of certain Markov process expectations for large time , 1975 .

[20]  S. Meyn,et al.  Large Deviations Asymptotics and the Spectral Theory of Multiplicatively Regular Markov Processes , 2005, math/0509310.

[21]  Qing Zhang,et al.  Continuous-Time Markov Chains and Applications , 1998 .

[22]  Thordur Runolfsson,et al.  Model reduction of nonreversible Markov chains , 2007, 2007 46th IEEE Conference on Decision and Control.

[23]  W. Huisinga,et al.  Metastability and Dominant Eigenvalues of Transfer Operators , 2006 .

[24]  M. Freidlin,et al.  Random Perturbations of Dynamical Systems , 1984 .

[25]  S. Meyn,et al.  Phase transitions and metastability in Markovian and molecular systems , 2004 .

[26]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[27]  Herbert A. Simon,et al.  Aggregation of Variables in Dynamic Systems , 1961 .

[28]  Andrew L. Rukhin,et al.  Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach , 2001, Technometrics.

[29]  P. Deuflharda,et al.  Identification of almost invariant aggregates in reversible nearly uncoupled Markov chains , 2000 .

[30]  Sean P. Meyn,et al.  Shannon meets Bellman: Feature based Markovian models for detection and optimization , 2008, 2008 47th IEEE Conference on Decision and Control.

[31]  Sanjay Lall,et al.  Model reduction, optimal prediction, and the Mori-Zwanzig representation of Markov chains , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[32]  M. Fiedler.  A property of eigenvectors of nonnegative symmetric matrices and its application to graph theory , 1975 .

[33]  Robert E. Mahony,et al.  Lumpable hidden Markov models-model reduction and reduced complexity filtering , 2000, IEEE Trans. Autom. Control..

[34]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[35]  Fan Chung,et al.  Spectral Graph Theory , 1996 .

[36]  J. A. Fill.  Eigenvalue bounds on convergence to stationarity for nonreversible markov chains , 1991 .

[37]  Sridhar Mahadevan,et al.  Value Function Approximation with Diffusion Wavelets and Laplacian Eigenfunctions , 2005, NIPS.

[38]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[39]  Fady Alajaji,et al.  The Kullback-Leibler divergence rate between Markov sources , 2004, IEEE Transactions on Information Theory.

[40]  John G. Kemeny,et al.  Finite Markov chains , 1960 .

[41]  Prashant G. Mehta,et al.  Graph based multi-scale analysis of building system transport models , 2006, 2006 American Control Conference.

[42]  P. Rabinowitz.  Minimax methods in critical point theory with applications to differential equations , 1986 .

[43]  Sean P. Meyn,et al.  Model reduction for reduced order estimation in traffic models , 2008, 2008 American Control Conference.

[44]  Sean P. Meyn,et al.  An information-theoretic framework to aggregate a Markov chain , 2009, 2009 American Control Conference.