Divergence From Factorizable Distributions and Matroid Representations by Partitions

Maximization of the information divergence from any hierarchical log-linear model is studied. A new upper bound on the maximum is presented and its tightness analyzed. For the models given by the bases of a matroid, the latter is related to matroid representations by partitions or, equivalently, to ideal secret-sharing schemes. A new link between the divergence maximization, the maximum-likelihood principle, and secret sharing is established.

[1]  S. Haberman,et al.  The analysis of frequency data , 1974 .

[2]  J. F. C. Kingman,et al.  Information and Exponential Families in Statistical Theory , 1980 .

[3]  N. N. Chent︠s︡ov.  Statistical decision rules and optimal inference , 1982 .

[4]  L. Brown.  Fundamentals of statistical exponential families: with applications in statistical decision theory , 1986 .

[5]  James G. Oxley,et al.  Matroid theory , 1992 .

[6]  F. Matús.  PROBABILISTIC CONDITIONAL INDEPENDENCE STRUCTURES AND MATROID THEORY: BACKGROUND1 , 1993 .

[7]  John D. Lafferty,et al.  Inducing Features of Random Fields , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Frantisek Matús,et al.  Matroid representations by partitions , 1999, Discret. Math..

[9]  N. Čencov.  Statistical Decision Rules and Optimal Inference , 2000 .

[10]  Raymond W. Yeung,et al.  A First Course in Information Theory , 2002 .

[11]  N. Ay.  AN INFORMATION-GEOMETRIC APPROACH TO A THEORY OF PRAGMATIC STRUCTURING , 2002 .

[12]  N. Ay,et al.  On maximization of the information divergence from an exponential family , 2003 .

[13]  Imre Csiszár,et al.  Information projections revisited , 2000, IEEE Trans. Inf. Theory.

[14]  Imre Csiszár,et al.  Information Theory and Statistics: A Tutorial , 2004, Found. Trends Commun. Inf. Theory.

[15]  D. Geiger,et al.  On the toric algebra of graphical models , 2006, math/0608054.

[16]  Raymond W. Yeung,et al.  A First Course in Information Theory (Information Technology: Transmission, Processing and Storage) , 2006 .

[17]  Nihat Ay,et al.  Maximizing Multi-Information , 2007 .

[18]  F. Mat.  Two Constructions on Limits of Entropy Functions , 2007, IEEE Trans. Inf. Theory.

[19]  Frantisek Matús,et al.  Optimality conditions for maximizers of the information divergence from an exponential family , 2007, Kybernetika.

[20]  I. Csiszár,et al.  Generalized maximum likelihood estimates for exponential families , 2008 .

[21]  L. Satyanarayan,et al.  Cellular polypropylene polymer foam as air-coupled ultrasonic transducer materials , 2010, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.