Learning and robust learning of product distributions

In this paper we present some results about PAC-learning distributions with respect to the Kullback-Leibler Divergence. The hypothesis classes are distributions which are the product of conditional probabilities of at most k + 1 variables. We call this family of distribution classes CPCPk. Our main theorem shows that all classes in C’PCPk are robustly learnable. This generalizes ideas of Lewis and Chow et al., who investigated restricted classes in CPCPk and only proved a weak convergence result for classes in CPCP1. Our approach is to transform the learning problem to a structured combinatorial optimization problem. The second part of this paper develops efficient algorithms for such problems thereby guaranteeing efficient learnability. We show the efficient robust learnability of tree dependent distributions and develop a combinatorial algorithm for the efficient proper learnability of Markov expansions. This algrithm finds a maximal Hamiltonian path for certain classes of graphs. Moreover we discuss the efficient learnability of classes in CPCPk for k > 1. Here the combinatorial problems are more involved, but we state positive results for learning Chow(k) expansions in special cases.