Letter Recognition Using Holland-Style Adaptive Classifiers

Machine rule induction was examined on a difficult categorization problem by applying a Holland-style classifier system to a complex letter recognition task. A set of 20,000 unique letter images was generated by randomly distorting pixel images of the 26 uppercase letters from 20 different commercial fonts. The parent fonts represented a full range of character types including script, italic, serif, and Gothic. The features of each of the 20,000 characters were summarized in terms of 16 primitive numerical attributes. Our research focused on machine induction techniques for generating IF-THEN classifiers in which the IF part was a list of values for each of the 16 attributes and the THEN part was the correct category, i.e., one of the 26 letters of the alphabet. We examined the effects of different procedures for encoding attributes, deriving new rules, and apportioning credit among the rules. Binary and Gray-code attribute encodings that required exact matches for rule activation were compared with integer representations that employed fuzzy matching for rule activation. Random and genetic methods for rule creation were compared with instance-based generalization. The strength/specificity method for credit apportionment was compared with a procedure we call “accuracy/utility.”

[1]  Philip J. Stone,et al.  Experiments in induction , 1966 .

[2]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[3]  N. Charness,et al.  Aging and skilled problem solving. , 1981, Journal of experimental psychology. General.

[4]  Stewart W. Wilson.  Knowledge Growth in an Artificial Animal , 1985, ICGA.

[5]  John H. Holland,et al.  Escaping brittleness: the possibilities of general-purpose learning algorithms applied to parallel rule-based systems , 1995 .

[6]  J. David Schaffer,et al.  Representation and Hidden Bias: Gray vs. Binary Coding for Genetic Algorithms , 1988, ML.

[7]  George G. Robertson,et al.  Population Size in classifier Systems , 1988, ML.

[8]  Lashon B. Booker,et al.  Classifier systems that learn internal world models , 1988, Machine Learning.

[9]  John H. Holland,et al.  Induction: Processes of Inference, Learning, and Discovery , 1987, IEEE Expert.

[10]  David Zipser,et al.  Feature Discovery by Competive Learning , 1986, Cogn. Sci..

[11]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[12]  Stewart W. Wilson.  Classifier Systems and the Animat Problem , 1987, Machine Learning.

[13]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[14]  H. Simon,et al.  Perception in chess , 1973 .

[15]  A. D. D. Groot.  Thought and Choice in Chess , 1978 .

[16]  David L. Waltz,et al.  Toward memory-based reasoning , 1986, CACM.

[17]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[18]  Lawrence Davis,et al.  Classifier Systems with Hamming Weights , 1988, ML.

[19]  James A. Anderson,et al.  Cognitive and psychological computation with neural models , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[20]  Stewart W. Wilson.  Bid Competition and Specificity Reconsidered , 1988, Complex Syst..

[21]  Donald Michie,et al.  Expert systems in the micro-electronic age , 1979 .

[22]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[23]  J. R. Quinlan.  Discovering rules by induction from large collections of examples Intro-ductory readings in expert s , 1979 .