On Information-Theoretic Characterizations of Markov Random Fields and Subfields

Let <inline-formula> <tex-math notation="LaTeX">$X_{i}, i~\in V$ </tex-math></inline-formula> form a Markov random field (MRF) represented by an undirected graph <inline-formula> <tex-math notation="LaTeX">$G = (V,E)$ </tex-math></inline-formula>, and <inline-formula> <tex-math notation="LaTeX">$V'$ </tex-math></inline-formula> be a subset of <inline-formula> <tex-math notation="LaTeX">$V$ </tex-math></inline-formula>. We determine the smallest graph that can always represent the subfield <inline-formula> <tex-math notation="LaTeX">$X_{i}, i~\in V'$ </tex-math></inline-formula> as an MRF. Based on this result, we obtain a necessary and sufficient condition for a subfield of a Markov tree to be also a Markov tree. When <inline-formula> <tex-math notation="LaTeX">$G$ </tex-math></inline-formula> is a path so that <inline-formula> <tex-math notation="LaTeX">$X_{i}, i~\in V$ </tex-math></inline-formula> form a Markov chain, it is known that the <inline-formula> <tex-math notation="LaTeX">$I$ </tex-math></inline-formula>-Measure is always nonnegative (Kawabata and Yeung in 1992). We prove that Markov chain is essentially the only MRF such that the <inline-formula> <tex-math notation="LaTeX">$I$ </tex-math></inline-formula>-Measure is always nonnegative. By applying our characterization of the smallest graph representation of a subfield of an MRF, we develop a recursive approach for constructing information diagrams for MRFs. Our work is built on the set-theoretic characterization of an MRF (Yeung <italic>et al.</italic> in 2002).

[1]  Francesco M. Malvestuto.  A unique formal system for binary decompositions of database relations, probability distributions, and graphs , 1992, Inf. Sci..

[2]  T. Snijders.  Statistical Models for Social Networks , 2011 .

[3]  Raymond W. Yeung,et al.  Information Theory and Network Coding , 2008 .

[4]  J. Laurie Snell,et al.  Markov Random Fields and Their Applications , 1980 .

[5]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[6]  Judy H. Cho,et al.  Incorporating Biological Pathways via a Markov Random Field Model in Genome-Wide Association Studies , 2011, PLoS genetics.

[7]  J. Pearl,et al.  Logical and Algorithmic Properties of Conditional Independence and Graphical Models , 1993 .

[8]  Hamid Krim,et al.  A Generalized Markov Graph Model: Application to Social Network Analysis , 2013, IEEE Journal of Selected Topics in Signal Processing.

[9]  A. Cohen.  On random fields , 1967 .

[10]  Chuanyi Ji,et al.  Randomized and Distributed Self-Configuration of Wireless Networks: Two-Layer Markov Random Fields and Near-Optimality , 2010, IEEE Transactions on Signal Processing.

[11]  Judea Pearl,et al.  GRAPHOIDS: A Graph-based logic for reasoning about relevance relations , 1985 .

[12]  Raymond W. Yeung,et al.  A framework for linear information inequalities , 1997, IEEE Trans. Inf. Theory.

[13]  J. M. Hammersley,et al.  Markov fields on finite graphs and lattices , 1971 .

[14]  Francesco M. Malvestuto,et al.  Comment on "A unique formal system for binary decompositions of database relations, probability distributions, and graphs" , 1992, Inf. Sci..

[15]  Tsutomu Kawabata,et al.  The structure of the I-measure of a Markov chain , 1992, IEEE Trans. Inf. Theory.

[16]  Milan Studený,et al.  Probabilistic conditional independence structures , 2006, Information science and statistics.

[17]  D. Edwards,et al.  Collapsibility and response variables in contingency tables , 1983 .

[18]  Steffen L. Lauritzen,et al.  Graphical models in R , 1996 .

[19]  Kayvan Sadeghi.  Stable mixed graphs , 2011, 1110.4168.

[20]  Roozbeh Farahbod,et al.  Adaptive routing in mobile ad hoc networks based on decision aid approach , 2010, MobiWac '10.

[21]  Raymond W. Yeung,et al.  A new outlook of Shannon's information measures , 1991, IEEE Trans. Inf. Theory.

[22]  Hongzhe Li,et al.  A hidden Markov random field model for genome-wide association studies. , 2010, Biostatistics.

[23]  Kayvan Sadeghi,et al.  Marginalization and Conditioning for LWF Chain Graphs , 2014, 1405.7129.

[24]  Hu Kuo Ting,et al.  On the Amount of Information , 1962 .

[25]  José M. F. Moura,et al.  Big Data Analysis with Signal Processing on Graphs: Representation and processing of massive data sets with irregular structure , 2014, IEEE Signal Processing Magazine.

[26]  Hongzhe Li,et al.  A Markov random field model for network-based analysis of genomic data , 2007, Bioinform..

[27]  P. Strevens.  Iii , 1985 .

[28]  L.E. Doyle,et al.  Ad hoc networking, markov random fields, and decision making , 2006, IEEE Signal Processing Magazine.

[29]  John P. Moussouris.  Gibbs and Markov random systems with constraints , 1974 .

[30]  Raymond W. Yeung,et al.  Information-theoretic characterizations of conditional mutual independence and Markov random fields , 2002, IEEE Trans. Inf. Theory.

[31]  Ouissem Ben Fredj.  A realistic graph-based alert correlation system , 2015, Secur. Commun. Networks.

[32]  G. Varoquaux,et al.  Markov models for fMRI correlation structure: Is brain functional connectivity small world, or decomposable into networks? , 2012, Journal of Physiology-Paris.

[33]  R. Yeung,et al.  On the rate-distortion region for multiple descriptions , 2000, 2000 IEEE International Symposium on Information Theory (Cat. No.00CH37060).