Genetic algorithms and machine learning

One approach to the design of learning systems is to extract heuristics from existing adaptive systems. Genetic algorithms are heuristic learning models based on principles drawn from natural evolution and selective breeding. Some features that distinguish genetic algorithms from other search methods are: A population of structures that can be interpreted as candidate solutions to the given problem; The competitive selection of structures for reproduction, based on each structure's tness as a solution to the given problem; Idealized genetic operators that alter the selected structures in order to create new structures for further testing. In many applications, these features enable the genetic algorithm to rapidly improve the average tness of the population and to quickly identify the high performance regions of very complex search spaces. In practice, genetic algorithms may be combined with local search techniques to create a high-performance hybrid search algorithm. This talk provides a survey of recent advances in the application of genetic algorithms to problems in machine learning. Although many genetic algorithm applications have been in the areas of function optimization, parameter tuning, scheduling and other combinatorial problems [1], genetic algorithms have also been applied to many traditional machine learning problems, including concept learning from examples, learning weights for neural nets, and learning rules for sequential decision problems. At NRL, we investigate many aspects of genetic algorithms, ranging from the study of alternative selection policies [6] and crossover operators [3, 12], to performance studies of genetic algorithms for optimization in non-stationary environments [8]. Much of our e ort has been devoted to the development of practical learning systems that use genetic algorithms to learn strategies for sequential decision problems [5]. In our Samuel system [7], the \chromosome" of the genetic algorithm represents a set of condition-action rules for controlling an autonomous vehicle or a robot. The tness of a rule set is measured by evaluating the performance of the resulting control strategy on a simulator. This system has successfully learned highly e ective strategies for several tasks, including evading a predator, tracking a prey, seeking a goal while avoiding obstacles, and defending a goal from threatening agents. As these examples show, we have a high level of interest in learning in multi-agent environments in which the behaviors of the external agents are not easily characterized by the learner. We have found that genetic algorithms provide an e cient way to learn strategies that take advantage of subtle regularities in the behavior of opposing agents. We are now beginning to investigate the more general case in which the behavior of the external agents changes over time. In particular, we are interested in learning competitive strategies against an opponent that is itself a learning agent. This is, of course, the usual situation in natural environments in which multiple species compete for survival. Our initial studies lead us to expect that genetic learning systems can successfully adapt to changing environmental conditions. While the range of applications of genetic algorithms continues to grow more rapidly each year, the study of the theoretical foundations is still in an early stage. Holland's early work [9] showed that a simple form of genetic algorithm implicitly estimates the utility of a vast number of distinct subspaces, and allocates future trials accordingly. Speci cally, let H be a hyperplane in the representation space. For example, if the structures are represented by six binary features, then the hyperplane denoted by H =0#1### consists of all structures in which the rst feature is absent and the third feature is present. Holland showed that the expected number of samples (o spring) allocated to a hyperplane H at time t + 1 is given by: M (H; t+ 1) M (H; t) f (H; t)