Mixtures of Kikuchi Approximations

Mixtures of distributions concern modeling a probability distribution by a weighted sum of other distributions. Kikuchi approximations of probability distributions follow an approach to approximate the free energy of statistical systems. In this paper, we introduce the mixture of Kikuchi approximations as a probability model. We present an algorithm for learning Kikuchi approximations from data based on the expectation-maximization (EM) paradigm. The proposal is tested in the approximation of probability distributions that arise in evolutionary computation.

[1]  Roberto Santana.  A Markov Network Based Factorized Distribution Algorithm for Optimization , 2003, ECML.

[2]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[3]  Michael I. Jordan,et al.  Learning with Mixtures of Trees , 2001, J. Mach. Learn. Res..

[4]  J. A. Lozano,et al.  Properties of Kikuchi approximations constructed from clique based decompositions , 2022 .

[5]  Dirk Thierens,et al.  Multi-objective optimization with diversity preserving mixture-based iterated density estimation evolutionary algorithms , 2002, Int. J. Approx. Reason..

[6]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[7]  R. Kikuchi.  A Theory of Cooperative Phenomena , 1951 .

[8]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[9]  Roberto Santana,et al.  Estimation of Distribution Algorithms with Kikuchi Approximations , 2005, Evolutionary Computation.

[10]  Neil D. Lawrence,et al.  Approximating Posterior Distributions in Belief Networks Using Mixtures , 1997, NIPS.

[11]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[12]  Pedro Larrañaga,et al.  Globally Multimodal Problem Optimization Via an Estimation of Distribution Algorithm Based on Unsupervised Learning of Bayesian Networks , 2005, Evolutionary Computation.

[13]  I. Bratko,et al.  Kikuchi-Bayes: Factorized Models for Approximate Classification in Closed Form , 2004 .

[14]  R. Santana,et al.  The mixture of trees Factorized Distribution Algorithm , 2001 .

[15]  William T. Freeman,et al.  Constructing free-energy approximations and generalized belief propagation algorithms , 2005, IEEE Transactions on Information Theory.

[16]  Stephen A. Cook,et al.  The complexity of theorem-proving procedures , 1971, STOC.