A Wrapper Approach for Constructive Induction

Inductive algorithms rely strongly on their representational biases. Representational inadequacy can be mitigated by constructive induction. This paper introduces the notion of a relative gain measure and describes a new constructive induction algorithm (GALA) which is independent of the learning algorithm. GALA generates a small number of new boolean attributes from existing boolean, nominal or real-valued attributes. Unlike most previous research on constructive induction, our methods are designed as preprocessing step before standard machine learning algorithms are applied. We present results which demonstrate the e ectiveness of GALA on both arti cial and real domains for both symbolic and subsymbolic learners. For symbolic learners, we used C4.5 and CN2. For subsymbolic learners, we used perceptron and backpropagation. In all cases, the GALA preprocessor increased the performance of the learning algorithm.

[1]  Benjamin W. Wah,et al.  Principled Constructive Induction , 1989, IJCAI.

[2]  Steven W. Norton.  Generating Better Decision Trees , 1989, IJCAI.

[3]  Michael J. Shaw,et al.  Complex Concept Acquisition through Directed Search and Feature Caching , 1993, IJCAI.

[4]  Larry A. Rendell,et al.  A Scheme for Feature Construction and a Comparison of Empirical Methods , 1991, IJCAI.

[5]  Larry A. Rendell,et al.  Lookahead Feature Construction for Learning Hard Concepts , 1993, International Conference on Machine Learning.

[6]  Peter Clark,et al.  Rule Induction with CN2: Some Recent Improvements , 1991, EWSL.

[7]  Christopher J. Matheus,et al.  The Need for Constructive Induction , 1991, ML.

[8]  Thomas G. Dietterich,et al.  Inductive Learning of Structural Descriptions: Evaluation Criteria and Comparative Review of Selected Methods , 1981, Artif. Intell..

[9]  Thomas G. Dietterich,et al.  Selecting Appropriate Representations for Learning from Examples , 1986, AAAI.

[10]  David W. Aha,et al.  Incremental Constructive Induction: An Instance-Based Approach , 1991, ML.

[11]  Carl Myers Kadie.  Quantifying the Value of Constructive Induction, Knowledge, and Noise Filtering on Inductive Learning , 1991, ML.

[12]  Larry A. Rendell,et al.  Constructive Induction On Decision Trees , 1989, IJCAI.

[13]  D. Haussler,et al.  Boolean Feature Discovery in Empirical Learning , 1990, Machine Learning.

[14]  Paul E. Utgoff,et al.  Automatic Feature Generation for Problem Solving Systems , 1992, ML.

[15]  Peter Clark,et al.  The CN2 induction algorithm , 2004, Machine Learning.

[16]  Larry A. Rendell,et al.  Using Multidimensional Projection to Find Relations , 1995, ICML.

[17]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[18]  Larry A. Rendell,et al.  Improving the Design of Induction Methods by Analyzing Algorithm Functionality and Data-Based Concept Complexity , 1993, IJCAI.