An incremental learning algorithm for constructing Boolean functions from positive and negative examples

This paper introduces an incremental algorithm for learning a Boolean function from examples. The functions are constructed in the disjunctive normal form (DNF) or the conjunctive normal form (CNF) and emphasis is placed in inferring functions with as few clauses as possible. This incremental algorithm can be combined with any existing algorithm that infers a Boolean function from examples. In this paper it is combined with the one clause at a time (OCAT) approach (Comput. Oper. Res. 21(2) (1994) 185) and (J. Global Optim. 5(1) (1994) 64) which is a non-incremental learning approach. An extensive computational study was undertaken to assess the performance characteristics of the new approach. As examples, we used binary vectors that represent text documents from different categories from the TIPSTER collection. The computational results indicate that the new algorithm is considerably more efficient and it derives more accurate Boolean functions. As it was anticipated, the Boolean functions (in DNF or CNF form) derived by the new algorithm are comprised by more clauses than the functions derived by the non-incremental approach.

[1]  Jay F. Nunamaker,et al.  Automatic concept classification of text from electronic meetings , 1994, CACM.

[2]  Ryszard S. Michalski,et al.  Selection of Most Representative Training Examples and Incremental Generation of VL1 Hypotheses: The Underlying Methodology and the Description of Programs ESEL and AQ11 , 1978 .

[3]  Philip J. Stone,et al.  Experiments in induction , 1966 .

[4]  J. Barnes.  Statistical Analysis for Engineers and Scientists: A Computer-Based Approach , 1994 .

[5]  Gerard Salton,et al.  Automatic Information Organization And Retrieval , 1968 .

[6]  Evangelos Triantaphyllou.  Inference of a minimum size boolean function from examples by using a new efficient branch-and-bound approach , 1994, J. Glob. Optim..

[7]  Paul E. Utgoff,et al.  Incremental Induction of Decision Trees , 1989, Machine Learning.

[8]  A. Soyster,et al.  An approach to guided learning of boolean functions , 1996 .

[9]  Alan Lester. Meyrowitz.  Foundations of Knowledge Acquisition: Machine Learning , 1993 .

[10]  Gerard Salton,et al.  Automatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer , 1989 .

[11]  Ryszard S. Michalski.  Knowledge Repair Mechanisms: Evolution vs Revolution , 1985 .

[12]  E. T RIANTAPHYLLOU,et al.  A Greedy Randomized Adaptive Search Procedure ( GRASP ) for Inferring Logical Clauses from Examples in Polynomial Time and some Extensions , 1998 .

[13]  Donna K. Harman,et al.  Overview of the Sixth Text REtrieval Conference (TREC-6) , 1997, Inf. Process. Manag..

[14]  John H. Holland,et al.  Induction: Processes of Inference, Learning, and Discovery , 1987, IEEE Expert.

[15]  David Haussler,et al.  The Probably Approximately Correct (PAC) and Other Learning Models , 1993 .

[16]  James F. Gimpel.  A Method of Producing a Boolean Function Having an Arbitrarily Prescribed Prime Implicant Table , 1965, IEEE Trans. Electron. Comput..

[17]  Robert K. Brayton,et al.  Logic Minimization Algorithms for VLSI Synthesis , 1984, The Kluwer International Series in Engineering and Computer Science.

[18]  Winfrid G. Schneeweiss,et al.  Boolean functions - with engineering applications and computer programs , 1989 .

[19]  Frank Harary,et al.  A formal system for information retrieval from files , 1970, Commun. ACM.

[20]  M. Karnaugh.  The map method for synthesis of combinational logic circuits , 1953, Transactions of the American Institute of Electrical Engineers, Part I: Communication and Electronics.

[21]  Susan Chipman,et al.  Foundations of Knowledge Acquisition , 1993 .

[22]  Douglas H. Fisher,et al.  A Case Study of Incremental Concept Induction , 1986, AAAI.

[23]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[24]  Bella Hass Weinberg,et al.  Introduction to indexing and abstracting , 1991, J. Am. Soc. Inf. Sci..

[25]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[26]  Toshihide Ibaraki,et al.  An Implementation of Logical Analysis of Data , 2000, IEEE Trans. Knowl. Data Eng..

[27]  Jeffrey C. Schlimmer.  Incremental Adjustment of Representations for Learning , 1987 .

[28]  Charles T. Meadow,et al.  Text information retrieval systems , 1992 .

[29]  Donna K. Harman,et al.  Overview of the Second Text REtrieval Conference (TREC-2) , 1994, HLT.

[30]  Evangelos Triantaphyllou,et al.  A Relationship Between CNF and DNF Systems Derivable from Examples , 1995, INFORMS J. Comput..

[31]  H. Chen,et al.  A machine learning approach to document retrieval: an overview and an experiment , 1994, 1994 Proceedings of the Twenty-Seventh Hawaii International Conference on System Sciences.

[32]  Soundar R. T. Kumara,et al.  Generating logical expressions from positive and negative examples via a branch-and-bound approach , 1994, Comput. Oper. Res..

[33]  Ryszard S. Michalski,et al.  Discovering Classification Rules Using variable-Valued Logic System VL1 , 1973, IJCAI.

[34]  Paul E. Utgoff,et al.  Decision Tree Induction Based on Efficient Tree Restructuring , 1997, Machine Learning.