Co-evolution for Problem Simplification

This paper explores a co-evolutionary approach applicable to difficult problems with limited failure/success performance feedback. Like familiar "predator-prey" frameworks this algorithm evolves two populations of individuals the solutions (predators) and the problems (prey). The approach extends previous work by rewarding only the problems that match their difficulty to the level of solut.ion competence. In complex problem domains with limited feedback, this "tractability constraint" helps provide an adaptive fitness gradient that effectively differentiates the candidate solutions. The algorithm generates selective pressure toward the evolution of increasingly competent solutions by rewarding solution generality and uniqueness and problem tractability and difficulty. Relative (inverse-fitness) and absolute (static objective function) approaches to evaluating problem difficulty are explored and discussed. On a simple control task, this co-evolutionary algorithm was found to have significant advantages over a genetic algorithm with either a static fitness fimction or a fitness fimction that changes on a hand-tuned schedule. 1 Theoretical Background Traditional evolutionary algorithms evaluate the fitness of an individual by evaluating its ability to minimize an objective function which is typically static and independent of the evolutionary algorithm. For example, if the goal is to evolve a posture controller for a robot, the fitness of an individual controller could be its success in minimizing movement in the robot body under a gravity load. In co-evolutionary algorithms, the fitness of an individual in the evolving population(s) depends on interactions with other individuals in the same generation. The problems (e.g. the forces on the robot) faced by individuals in a coevolutionary algorithnl are dynamic and are shaped by the algorithm itself. Extending the robot example, the situations that a robot faces (e.g. forces on the robot like gravity) could be co-evolving with the controllers such that the set of situations on which controllers are evaluated changes from generation to generation. 1.1 Co-evo[ution: Competition and Cooperation A growing body of research explores co-evolutionary approaches that capitalize on this dynamic quality (for review, see Paredis, 1998) . This co-evolutionary work has largely concentrated on competitive interactions. The interactions can be between individuals that compete in a synunetric game-like context (Pollack et al., 1996; Sims, 1994; Rosin, 1997) or between populations of different types of individuals that compete in predator/prey type relationships (Hillis, 1991; Paredis, 1994b; Paredis, 1994a; Cliff & Miller, 1996; Juille & Pollack, 1998; Rosin, 1997; Rosin & Belew, 1996). In these cases, individuals are rewarded if they defeat the individuals with which they compete. These interactions can support "arms-races" in which the individualsforceeachotherto become increasingly competent. A fewstudieshaveinvestigated theroleof cooperation and how it can help solve some problems endemic to evolutionary methods, like the difficulty of choosing an appropriate encoding for the individuals (Paredis, 1995) and tile difficulty of decomposing composite problems (Jong & Potter, 1995). Other studies have found that a balance of cooperation and competition is necessary t.o prevent evohltionary algorithms from getting trapped in h)cal minima, or "Mediocre Stable States" (Ficici, 1995). 1.2 The Current Approach The approach outlined in this paper has features of both competitive and cooperati_v co-(,vohttionary approaches. The algorithn_ tries to ensure a tractable learning gradient for the solutions by rewarding only those problems on which at. least one solution was successful. The fitness of these tractable problems is proportional to their absolute and/or relative difficulty providing pressure for the solutions to hecome more generally competent. In practice, this requirement generates an initial simplification and gradual increase in problem difficulty over evolution. The aim is to select for problems that are on the edge of what is solvable by the current population of solutions, ensuring a useful fitness gradient throughout evolution. This requirement that the problem must be tractable has t)een relatively unstressed in the literature, with a couple notable exceptions. Rosin (1997) suggests a mechanism (the "Phantom Parasite") that rewards problems that are solvable by at least one solution. This mechanism will tend to allow easy problems to survive in a population of very difficult problems. Juille and Pollack (1998) use a domain specific approach to selecting for problem tractability by rewarding problems that. tend to be easier by an objective