When Prior Knowledge Hinders Learning

There are two general approaches to learning classification rules. Empirical learning programs operate by finding regularities among a group of training examples. Analytic learning systems use prior knowledge to explain the classification of examples and form a general description of the class of examples with the same explanation. 1989) have combined empirical and analytic learning methods. In such systems, the prior knowledge is used to bias the learner to prefer concepts that are consistent with the prior knowledge. This bias is intended to make a learner create concept descriptions that are more accurate classifiers than both the original prior knowledge (that serves as input to the analytic learning component) and the rules that would arise if only the empirical learning component were used. In previous work (Pazzani, Brunk, & Silverstein, 1991; Pazzani & Kibler, in press), we have described FOCL, a system that extends Quinlan's (1990) FOIL program in a number of ways, most significantly by adding a compatible explanation-based learning (EBL) component. Here, we describe a simple example deliberately constructed to show that prior knowledge can also hinder learning in FOCL as well as other combined learned systems. We analyze why this example presents problems for these systems and suggest an approach that allows FOCL to exploit this type of prior knowledge to facilitate learning. 1.1 FOIL FOIL learns classification rules by constructing a set of Horn Clauses in terms of a set of known operational predicates. Each clause body consists of a conjunction of literals that cover some positive and no negative examples. FOIL starts to learn a clause body by finding the literal with the maximum information gain 1 , and continues to add literals to the clause body until the clause does not cover any negative examples. After learning each clause, FOIL removes the positive examples covered by that clause from further consideration. The learning process terminates when all positive examples have been covered by a clause. 1.2 FOCL FOCL extends FOIL by including a compatible EBL component. This allows FOCL to take advantage of a given domain theory. When constructing a clause body, there are two ways that FOCL can add literals. First, it can create literals via the same empirical method used by FOIL. Second, it can create literals by operationalizing a target concept, i.e., a non-operational definition of the concept to be learned (Mitchell, Keller, & Kedar-Cabelli, 1986). FOCL uses the information-based evaluation …