Boolean Feature Discovery in Empirical Learning

We investigate the problem of learning Boolean functions with a short DNF representation using decision trees as a concept description language. Unfortunately, Boolean concepts with a short description may not have a small decision tree representation when the tests at the nodes are limited to the primitive attributes. This representational shortcoming may be overcome by using Boolean features at the decision nodes. We present two new methods that adaptively introduce relevant features while learning a decision tree from examples. We show empirically that these methods outperform a standard decision tree algorithm for learning small random DNF functions when the examples are drawn at random from the uniform distribution.

[1]  L. Beach.  Cue probabilism and inference behavior. , 1964 .

[2]  Tom M. Mitchell,et al.  Acquisition of Appropriate Bias for Inductive Concept Learning , 1982, AAAI.

[3]  Jaime G. Carbonell,et al.  An Overview of Machine Learning , 1983 .

[4]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[5]  Thomas G. Dietterich,et al.  Selecting Appropriate Representations for Learning from Examples , 1986, AAAI.

[6]  Stephen Muggleton,et al.  Duce, An Oracle-based Approach to Constructive Induction , 1987, IJCAI.

[7]  Ronald L. Rivest,et al.  Learning decision lists , 2004, Machine Learning.

[8]  David Haussler,et al.  Occam's Razor , 1987, Inf. Process. Lett..

[9]  J. C. Schlimmer,et al.  Concept acquisition through representational adjustment , 1987 .

[10]  J. Ross Quinlan,et al.  Generating Production Rules from Decision Trees , 1987, IJCAI.

[11]  M. Kearns,et al.  Recent Results on Boolean Concept Learning , 1987 .

[12]  J. Ross Quinlan,et al.  An Empirical Comparison of Genetic and Decision-Tree Classifiers , 1988, ML.

[13]  David Haussler,et al.  Quantifying Inductive Bias: AI Learning Algorithms and Valiant's Learning Framework , 1988, Artif. Intell..

[14]  Giulia Pagallo,et al.  Learning DNF by Decision Trees , 1989, IJCAI.

[15]  Larry A. Rendell,et al.  Feature construction: an analytic framework and an application to decision trees , 1990 .

[16]  J. Ross Quinlan,et al.  Simplifying decision trees , 1987, Int. J. Hum. Comput. Stud..

[17]  Peter Clark,et al.  The CN2 Induction Algorithm , 1989, Machine Learning.

[18]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[19]  Stewart W. Wilson.  Classifier Systems and the Animat Problem , 1987, Machine Learning.

[20]  V. Vapnik.  Estimation of Dependences Based on Empirical Data , 2006 .

[21]  K. Murphy,et al.  Overview of Machine Learning , 2022, International Journal of Advanced Research in Science, Communication and Technology.