A new approach to genetic-based automatic feature discovery

Systems which take raw data and categorize them into discrete classes are ubiquitous in computer science, having applications in elds such as vision, expert systems, and game playing. These systems work by extracting features from the data and then combining the values of the features to form a judgement. While much work has been done on ways to automatically combine feature values, the task of automatic discovery of these features is recognized to be much more di cult, and so has become one of the holy grails of machine learning. Classi er systems, an outgrowth of genetic algorithms, seemed a promising approach to automatic feature discovery, but it is di cult to get the full power of the classi er system from existing implementations. This thesis simpli es the classi er system into a variant of the genetic algorithm, called the Population Genetic Algorithm (PGA). PGAs are used to automatically discover features for tic-tac-toe and checkers endgame positions, and these features are automatically combined using Bayesian statistics to classify each position as won, lost, or drawn. The theoretical maximum performance of the PGAs is determined by using an exhaustive enumeration technique to serve as a baseline comparison. The results indicate that while PGAs can be made to perform at near-optimal levels, the optimal solution is insu cient to perfectly classify any of the domains studied. Acknowledgements The following people were invaluable to me in the production of this thesis. My deepest thanks go to all of them: Jonathan Schae er, my supervisor, for his guidance, encouragement, and experience, Joe Culberson and Peter Dixon, my examining committee, for taking the time to read my thesis and o er their valuable suggestions, Andreas Junghanns, Yngvi Bjornsson, and Mark Brockington for their input and enthusiasm, and Edmund Dengler, who helped develop many of the initial ideas.