How Learning can guide evolution in hierarchical modular tasks - eScholarship

How learning can guide evolution in hierarchical modular tasks Janet Wiles (janetw@csee.uq.edu.au) School of Psychology and School of Computer Science and Electrical Engineering University of Queensland, Qld 4072 Australia Bradley Tonkes (btonkes@csee.uq.edu.au) James R. Watson (jwatson@csee.uq.edu.au) School of Computer Science and Electrical Engineering University of Queensland, Qld 4072 Australia Abstract This paper addresses the problem of how and when learn- ing is an aid to evolutionary search in hierarchical modu- lar tasks. It brings together two major areas of research in evolutionary computation, the performance of evolution- ary algorithms on hierarchical modular tasks, and the role of learning in evolutionary search, known as the Bald- win effect. A new task called the jester’s cap is pro- posed, formed by adding learning to Watson, Hornby and Pollack’s Hierarchical-If-and-only-If, function, using the simple guessing framework of Hinton and Nowlan’s Baldwin effect simulations. Whereas Hinton and Nowlan used a task with a single fitness peak, ideally suited to learning, the jester’s cap is a hierarchical task that has two major fitness peaks and multiple sub-peaks. We con- ducted a series of simulations to explore the effect of dif- ferent amounts of learning on the jester’s cap. The sim- ulations demonstrate that learning aids evolution only in search spaces in which the simplest level of modules are difficult to find. The learning mechanism explores lo- cal regions of the search space, while crossover explores neighborhoods in higher-order modular spaces. Introduction This paper addresses the problem of how and when learn- ing is an aid to evolutionary search in hierarchical mod- ular tasks. It brings together two major areas of research in evolutionary computation (EC), the performance of evolutionary algorithms (EAs) on hierarchical modular tasks, and computational models of the role of learning in evolutionary search, known as the Baldwin effect. We begin with a brief review of modular tasks that have been proposed to explore the performance of evo- lutionary algorithms, and then briefly describe the Bald- win effect. We then describe a specific task, the jester’s cap, that incorporates learning into a hierarchical mod- ular task. In many simulation tasks, learning is costly and does not improve the performance of an evolutionary algorithm (French and Messinger, 1994; Mayley, 1996; Pereira et al., 2000). This study is as much an investiga- tion of things that don’t learn, as of ones that do. There are many types of EAs, and the field of evolu- tionary computation is still determining features of prob- lems that are easy or hard for a particular class of EA, and the conditions under which such algorithms will perform better than other search techniques. In evolutionary com- putation, characterization of an EA’s performance con- cerns not just optimization per se, but the behaviors of populations as a whole, reflecting their original motiva- tions as models (albeit abstract ones) of real evolutionary processes. Some of the oldest and most popular techniques for evolutionary search are genetic algorithms (GAs), which use crossover as their major search technique. Originally developed by Holland (1992), their efficacy is thought to be based on groups of genes acting together as mod- ules (or building blocks, to use Holland’s original ter- minology), and have been studied extensively since (for general introductions see Goldberg, 1989 and Mitchell, A variety of modular tasks have been proposed to study the conditions under which GAs outperform com- parable search techniques. The most widely known of these are the Royal Road (RR) problems introduced by Mitchell et al. (1992). However, some forms of hill- climbers were found to easily outperform the GA, and a variety of tasks that incorporate deceptive elements have been defined (s.a., RR4 by Mitchell et al., 1994; HDF by Pelikan and Goldberg, 2000; hdf by Holland, 2000). An alternative approach to incorporating deceptive elements is to define a fitness function with two or more conflicting maxima. Watson et al. (1998) defined Hierarchical-If-and-only-If (H-IFF) as such a function. H-IFF is a simple function that is hierarchical, modular, is not searchable by mutation, but is amenable to search by crossover. Its defining characteristics are two fitness peaks at opposing ends of the search space. Combina- tions of the sub-components that comprise each level of the competing hierarchies cause many sub-optimal peaks and consequently many local minima (see below for the complete definition). Before proceeding further with the computational as- pects, it is worthwhile considering the relevance of mod- ule building to many areas of cognitive science. The role of modules in evolution has long been recognized (e.g., Dawkins, 1986). In evolutionary psychology there is a particularly strong interest in modules, in part due to Tooby and Cosmides (1994) claims that humans have behavioral modules analogous to other mental functions. By studying building block problems, we are consider- ing the types of processes that allow species to evolve varieties of modules, and their combination into com- plex mental organs. For example, echolocation in bats requires both the ability to emit and to receive high fre-