Shift of bias for inductive concept learning

We identify and examine the fundamental role that bias plays in inductive concept learning. Bias is the set of all influences, procedural or declarative, that causes a concept learner to prefer one hypothesis to another. Much of the success of concept learning programs to date results from the program's author having provided the learning program with appropriate bias. To date there has been no good mechanical method for shifting from one bias to another that is better. Instead, the author of a learning program has himself had to search for a better bias. The program author manually generates a bias, from scratch or by revising a previous bias, and then tests it in his program. If the author is not satisfied with the induced concepts, then he repeats the manual-generate and program-test cycle. If the author is satisfied, then he deems his program successful. Too often, he does not recognize his own role in the learning process. Our thesis is that search for appropriate bias is itself a major part of the learning task, and that we can create mechanical procedures for conducting a well-directed search for an appropriate bias. We would like to understand better how a program author goes about doing his search for appropriate bias. What insights does he have? What does he learn when he observes that a particular bias produces poor performance? What domain knowledge does he apply? We explore the problem of mechanizing the search for appropriate bias. To that end, we develop a framework for a procedure that shifts bias. We then build two instantiations of the procedure in a program called STABB, which we then incorporate in the LEX learning program. One, called "least disjunction", uses simple syntactic manipulation, and the other, called "constraint back propagation" uses analytic deduction. We report experiments with the implementations that both demonstrate the usefulness of the framework, and uncover important issues for this kind of learning.