Genetic Convergence in a Species of Evolved Robot Control Architectures

We analyse how the project of evolving neural network controllers for autonomous visually guided robots is signi cantly di erent from the usual function optimisation problems standard genetic algorithms are asked to tackle The need to have open ended increase in complexity of the controllers to allow for an inde nite number of new tasks to be incrementally added to the robot s capabilities in the long term means that genotypes of arbitrary length need to be allowed This results in populations being genetically converged as new tasks are added and needs a change to usual genetic algorithm practices Results of successful runs are shown and the population is analysed in terms of genetic convergence and movement in time across sequence space Introduction In the context of our ongoing project to evolve neural networks which act as controllers for visually guided autonomous robots some basic questions as to the nature of evolution have to be faced Can such a project be treated as a function optimisation problem for which standard genetic algorithms GAs have been designed If as will be argued below there is a signi cant di erence between evolution and optimisation then what changes in GAs are needed to deal with this In answering these questions particular attention will be paid in this paper to issues of genetic convergence which in standard GAs is usually taken to signal the end of the road Brief details of the networks which act as controllers and of the robots themselves which are required to perform simple navigational tasks using vision and touch sensors will be given here for fuller information see the papers cited below A particular run which results in successful behaviour will be analysed in terms of the movement of the population across sequence space Evolution versus Optimisation Genetic Algorithms have been focused in such a concentrated fashion on function optimisa tion problems that for instance when De Jong presented a paper bringing to peoples attention the fact that Holland s motivation for his initial GA work was the design and implementation of robust adaptive systems a much broader context this was greeted with some surprise and scepticism by the audience De Jong stressed that it was a fallacy to equate GAs with function optimisation The generic GA was not designed to solve any particular problem but was rather a high level simulation of a biological adaptive system Darwinian evolution One of many ways to think of evolution is as a strategy for exploration and traversal of complex time varying tness landscapes In natural evolution making the rather large assumption that tness can be unproblematically de ned there remains the question of what are to be the horizontal dimensions of such a landscape They may be phenotypic characteristics or genotypic ones whichever they are they can only be treated as well speci ed and meaningful dimensions in the short term for just so long as changes in the population treated as moving across this landscape are not too radical There is no sensible single tness landscape which can simultaneously cater for apes jelly sh bacteria and self reproducing RNA molecules at the very origin of life although individual landscapes may be usefully posited for each one of these Function optimisation can of course use tness landscape language But the landscape is always fully speci ed by the speci c function being optimised and the problem is usually to nd the global optimum or some near optima of the whole landscape In contrast a tness landscape in evolution can only be speci ed with reference to some current population and their genetic or phenotypic characteristics and such a population will inevitably be already situated in some con ned region of this landscape Hence in so far as any question is being asked in evolution it is not what is best but rather where shall we go to from here In the context of robotics it has been suggested that the design by hand of the control systems for autonomous robots is reaching the limits of feasibility and that the only hope of future progress is through some evolutionary process Although current practice in GAs is an obvious starting place for establishing some such evolutionary process it is indeed evolution and not optimisation that is required It is proposed that a or several species of robots or robot control architectures should be evolved in an incremental fashion As each new task is added to the speci cation the starting place should be the current converged population rather than a fresh initial random spread SAGA principles Initial tasks for an autonomous robot whose architecture is genetically determined may be rigorously speci ed but it is not possible to specify in advance what future tasks may be inde nitely added Since there must be some at least loose correlation between the complexity of such an architecture and the length of a genotype which determines it then an evolutionary algorithm must be able to deal with genotypes of arbitrary lengths This lead to the development of SAGA Species Adaptation Genetic Algorithms principles in B A Hamming distance d Figure Mutation allows a population to explore along ridges towards potentially higher hills in the tness landscape This picture is potentially misleading in high dimensional landscapes A B Figure as there is not just a sin gle shortest distance between two points Hamming distance d apart in binary geno type sequence space there are d shortest routes and far more than this that are nearly as short brie y introduced and summarised below The immediate questions which raise themselves are How if at all can Holland s Schema Theorem be accommodated How should genetic operators which allow change in genotype length be handled And how should recombination be done between genotypes of di erent lengths If a coding from genotype to phenotype is chosen which allows inde nite increase in the length of the former associated with inde nite increase in complexity of the latter then the notions of schemata needed for Holland s Schema Theorem do not work Since the class of all genotypes with speci ed xed values for particular alleles is now in nite in size whereas normally with xed length genotypes it is of large but nite size the concept of an average tness for the members of this class becomes highly questionable The following route to partially reconciling the Schema Theorem with arbitrary length genotypes is rather devious and starts with a detour As discussed more fully in for all practical purposes evolution requires the tness landscape to be not too rugged for an explanation of the ultimately dead end nature of adaptation on fully rugged landscapes see For a landscape to be fairly smooth this implies that points close together in horizontal distance should in general be reasonably correlated in vertical distance i e tness If the horizontal dimensions refer to genotype space then immediate neighbours are those that can be reached in a single genetic operation such as mutation of a single bit If one also includes as neighbours those that can be reached by application of a genetic operator that changes genotype length then one will need to restrict those changes to ones that do not very often change tness by an arbitrarily large amount the landscape should not contain too many precipices As explained at the end of the next section this virtually eliminates the chance of being trapped on some local optimum If one can characterise the tness of the whole as composed of the sum of tness contributions determined by separate parts of the genotype genes if you like with a reasonable degree of epistatic interaction between these parts this smoothness requirement translates into one that the genotype length should not change by an enormous amount in any one genetic operation That is any change of length should only be slight the rst SAGA principle Of course small changes in the short term can build up to arbitrarily large changes in the arbitrarily long term But if in the long term the maximum genotype length of a population increases say from g to G then all the members of the later population will be descended from some of the earlier population despite the g dimensional earlier search space being minute in comparison to the later G dimensional one It follows that all bar perhaps the very original populations over evolutionary time scales must be genetically converged Not only must any changes in length be gradual but all the lengths within a population will be very similar and will be genetically converged at corresponding loci a species This has consequences for allowable genetic operators for recombination primarily that such an operator must produce o spring with genotypes of similar not necessarily identical lengths to their parents and homologous segments must be swapped Returning from this detour to see how the Schema Theorem is rescued it turns out that in these particular circumstances of only gradual change small nite bounds can in practice be put on genotype lengths in the short term and hence the class of all members of any given schema is now nite in size Convergence and Mutation The most visible di erence between genotypes under evolution and those in a standard GA for function optimisation is that at all times bar perhaps the very start the population is virtually genetically converged In a standard GA this is usually considered the end of the story The received folklore is that recombination is the driving force for genetic search and mutation is only a background operator To quote from Clearly the O n estimate for implicit parallelism is based on a diverse popu lation where many schemata are represented However as exponential allocation of observed best schemata accrues one can expect that the number of building blocks processed will decrease This is an inevitable consequence of convergence in the vanilla avour GA outlined above After convergence the GA popula tion will be compo