Model reduction of Markov chains via low-rank approximation

This paper is concerned with the model reduction for Markov chain models. The goal is to obtain a low-rank approximation to the original Markov transition matrix. A nuclear-norm regularized optimization problem is proposed for this purpose, in which the Kullback-Leibler divergence rate is used to measure the similarity between two Markov chains, and the nuclear norm is used to approximate the rank function. An efficient iterative optimization algorithm is developed to compute the solution to the regularized problem. The effectiveness of this approach is demonstrated via numerical examples.

[1]  Sean P. Meyn,et al.  An information-theoretic framework to aggregate a Markov chain , 2009, 2009 American Control Conference.

[2]  Fady Alajaji,et al.  The Kullback-Leibler divergence rate between Markov sources , 2004, IEEE Transactions on Information Theory.

[3]  Ian R. Petersen,et al.  Probabilistic distances between finite-state finite-alphabet hidden Markov models , 2005, IEEE Transactions on Automatic Control.

[4]  Sean P. Meyn,et al.  Optimal Kullback-Leibler Aggregation via Spectral Theory of Markov Chains , 2011, IEEE Transactions on Automatic Control.

[5]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[6]  Tiejun Li,et al.  Optimal partition and effective dynamics of complex networks , 2008, Proceedings of the National Academy of Sciences.

[7]  Gabriele Steidl,et al.  Deblurring Poissonian images by split Bregman techniques , 2010, J. Vis. Commun. Image Represent..

[8]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[9]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[10]  S. Meyn,et al.  Phase transitions and metastability in Markovian and molecular systems , 2004 .

[11]  Herbert A. Simon,et al.  Aggregation of Variables in Dynamic Systems , 1961 .

[12]  Mathukumalli Vidyasagar,et al.  Reduced-order modeling of Markov and hidden Markov processes via aggregation , 2010, 49th IEEE Conference on Decision and Control (CDC).

[13]  John Odentrantz,et al.  Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues , 2000, Technometrics.

[14]  J. Moreau.  Fonctions convexes duales et points proximaux dans un espace hilbertien , 1962 .