On the development of inductive learning algorithms: generating flexible and adaptable concept representations

Vast amount of research in machine learning has focused on creating new algorithms stemming from reenements to existing learning models (e.g., neural nets, decision trees, instance-based learners, rule-based systems, bayesian estimators). This is convenient when a model M already exists exhibiting satisfactory performance over the class of domains of study S. It may occur, however, that no model over S performs adequately; the deenition of a new learning model is then necessary. The construction of a new learning model cannot rely on assumptions common to a process based on reenements alone: that a global evaluation metric is enough to guide the development process (e.g., a comparison based on predictive accuracy alone), and that the strategy of the model is always correct (inherent limitations in the strategy may exist). When a model is deened from scratch, it is recommended to adopt a functional view. This means we must rst deene the set of functionalities the new mechanism will perform (e.g., to cope with feature and class noise, to generate hypotheses amenable to interpretation and analysis, to elucidate feature interaction eeectively, etc.). After implementation, a functional decomposition analysis can assess how well the diierent components in the mechanism are able to carry out such functionalities. In the rst part of this thesis I outline basic steps for the development of new learning models. The outline conveys relevant information to the developer: 1) that the set of goal functionalities must be made clear from the beginning, and that the design and implementation must be oriented to accomplish such functionalities; 2) much can be learned from a functional decomposition analysis; by analyzing each component individually we can detect possible deeciencies; 3) it is vital to propose a design in which all components are made explicit, and thus amenable to modiication or replacement. iv To provide evidence supporting the importance of a functional view in the design of new learning models, I describe two experimental case studies. These studies serve to support the following ideas: that separating an algorithm into its constituent components is helpful to identify the individual contribution of each component; and that components of diierent nature can be integrated to achieve a desired functionality. The second part of this thesis deals with the design and implementation of a new learning algorithm: HCL (Hierarchical Concept Learner). This part has two main goals: to put into practice the basic steps for the development of …