Learning with Maximum-Entropy Distributions

We are interested in distributions which are derived as a maximum entropy distribution from a given set of constraints. More specifically, we are interested in the case where the constraints are the expectation of individual and pairs of attributes. For such a given maximum entropy distribution (with some technical restrictions) we develop an efficient learning algorithm for read-once DNF. We extend our results to monotone read-k DNF following the techniques of (Hancock & Mansour, 1991).

[1]  Leslie G. Valiant,et al.  Cryptographic Limitations on Learning Boolean Formulae and Finite Automata , 1993, Machine Learning: From Theory to Applications.

[2]  Leslie G. Valiant,et al.  Learning Boolean formulas , 1994, JACM.

[3]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[4]  Nimrod Megiddo,et al.  Constructing small sample spaces satisfying given constraints , 1993, SIAM J. Discret. Math..

[5]  Dana Angluin,et al.  When won't membership queries help? , 1991, STOC '91.

[6]  Alon Itai,et al.  Learnability with Respect to Fixed Distributions , 1991, Theor. Comput. Sci..

[7]  Javed A. Aslam,et al.  Improved Noise-Tolerant Learning and Generalized Statistical Queries , 1994 .

[8]  J. Kilian,et al.  On the complexity of statistical reasoning , 1995, Proceedings Third Israel Symposium on the Theory of Computing and Systems.

[9]  Michael Kearns,et al.  Efficient noise-tolerant learning from statistical queries , 1993, STOC.

[10]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[11]  Athanasios Papoulis,et al.  Probability, Random Variables and Stochastic Processes , 1965 .

[12]  Yishay Mansour,et al.  Learning monotone ku DNF formulas on product distributions , 1991, COLT '91.

[13]  Leslie G. Valiant,et al.  Computational limitations on learning from examples , 1988, JACM.

[14]  David Haussler,et al.  Equivalence of models for polynomial learnability , 1988, COLT '88.

[15]  C. R. Smith,et al.  Maximum-Entropy and Bayesian Methods in Inverse Problems , 1985 .

[16]  Balas K. Natarajan,et al.  On learning Boolean functions , 1987, STOC.

[17]  H. Balsters,et al.  Learnability with respect to fixed distributions , 1991 .

[18]  I. Miller.  Probability, Random Variables, and Stochastic Processes , 1966 .