Reverse HillclimbingGenetic Algorithms and the Busy Beaver Problem

This paper introduces a new analysis tool called {\it reverse hillclimbing}, and demonstrates how it can be used to evaluate the performance of a genetic algorithm. Using reverse hillclimbing, one can calculate the exact probability that hillclimbing will attain some point in a landscape. From this, the expected number of evaluations before the point is found by hillclimbing can be calculated. This figure can be compared to the average number of evaluations done by a genetic algorithm. This procedure is illustrated using the {\it Busy Beaver problem}, an interesting problem of theoretical importance in its own right. At first sight, a genetic algorithm appears to perform very well on this landscape, after examining only a vanishingly small proportion of the space. Closer examination reveals that the number of evaluations it performs to discover an optimal solution compares poorly with even the simples form of hillclimbing. Finally, several other uses for reverse hillclimbing are discussed.

[1]  A. Turing.  On Computable Numbers, with an Application to the Entscheidungsproblem. , 1937 .

[2]  A. Turing,et al.  On Computable Numbers, with an Application to the Entscheidungsproblem. A Correction , 1938 .

[3]  S. C. Kleene,et al.  Introduction to Metamathematics , 1952 .

[4]  Hao Wang,et al.  A Variant to Turing's Theory of Computing Machines , 1957, JACM.

[5]  C. Y. Lee.  Automata and finite automata , 1960 .

[6]  T. Rado.  On non-computable functions , 1962 .

[7]  Hao Wang,et al.  Words in the History of a Turing Machine with a Fixed Input , 1963, JACM.

[8]  Robert W. Coffin,et al.  Simulation of a Turing machine on a digital computer , 1963, AFIPS '63 (Fall).

[9]  R. W. Coffin,et al.  Simulation of biological cells by systems composed of string-processing finite automata , 1899, AFIPS '64 (Spring).

[10]  C. Ogilvy,et al.  Tomorrow's Math. , 1964 .

[11]  Solutions of restricted cases of the halting problem applied to the determination of particular values of a non-computable function , 1965 .

[12]  Shen Lin,et al.  Computer Studies of Turing Machine Problems , 1965, JACM.

[13]  Allen H. Brady.  The Conjectured Highest Scoring Machines for Rado's Σ(k) for the Value k = 4 , 1966, IEEE Trans. Electron. Comput..

[14]  Review: Tibor Rado, On a Simple Source for Non-Computable Functions , 1967 .

[15]  Algorithmen, rekursive Funktionen und formale Sprachen , 1968 .

[16]  An Uncompletable Function , 1968 .

[17]  David Pager.  On the Problem of Finding Minimal Programs for Tables , 1969, Inf. Control..

[18]  Jeffrey D. Ullman,et al.  Formal languages and their relation to automata , 1969, Addison-Wesley series in computer science and information processing.

[19]  Gregory J. Chaitin,et al.  On the difficulty of computations , 1970, IEEE Trans. Inf. Theory.

[20]  A. G. Bell.  Partitioning Integers in n Dimensions , 1970, Comput. J..

[21]  V. A. Kozmidiadi,et al.  Subclasses of Turing machines reducible to finite automata , 1972 .

[22]  B. Weimann,et al.  Untersuchungen über haltende Programme für Turing-Maschinen mit 2 Zeichen und bis zu 5 Befehlen , 1972, GI Jahrestagung.

[23]  Donald S. Lynn.  New Results for Rado's Sigma Function for Binary Turing Machines , 1972, IEEE Transactions on Computers.

[24]  A. Tanenbaum.  Computer recreations , 1973 .

[25]  David Pager.  Further Results on the Problem of Finding Minimal Length Programs for Decision Tables , 1974, JACM.

[26]  James P. Jones,et al.  Recursive Undecidability — An Exposition , 1974 .

[27]  Gregory J. Chaitin,et al.  Information-Theoretic Limitations of Formal Systems , 1974, JACM.

[28]  Robert P. Daley.  Non-Complex Sequences: Characterizations and Examples , 1974, SWAT.

[29]  Gregory J. Chaitin,et al.  Information-Theoretic Computational Complexity , 1974 .

[30]  George Boolos,et al.  Computability and logic , 1974 .

[31]  G. Chaitin.  Randomness and Mathematical Proof , 1975 .

[32]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[33]  Paul M. B. Vitányi.  On a problem in the collective behavior of automata , 1976, Discret. Math..

[34]  Fred C. Hennie,et al.  Introduction to Computability , 1977 .

[35]  Herbert B. Enderton,et al.  Elements of Recursion Theory , 1977 .

[36]  Howard P. Katseff,et al.  Several results in program size complexity , 1977, 18th Annual Symposium on Foundations of Computer Science (sfcs 1977).

[37]  Robert P. Daley.  On the Simplicity of Busy Beaver Sets , 1978, Math. Log. Q..

[39]  Frank S. Beckman.  Mathematical foundations of programming , 1980 .

[40]  Robert P. Daley.  Quantitative and Qualitative Information in Computations , 1980, Inf. Control..

[41]  Some rapidly growing functions , 1980 .

[42]  Robert P. Daley.  Retraceability, Repleteness and Busy Beaver Sets , 1981, MFCS.

[43]  Robert P. Daley.  Busy Beaver Sets and the Degrees of Unsolvability , 1981, J. Symb. Log..

[44]  Robert P. Daley.  Busy Beaver Sets: Characterizations and Applications , 1982, Inf. Control..

[45]  T. R. S. Walsh.  The busy beaver on a one-way infinite tape , 1982, SIGA.

[46]  Péter Gács,et al.  On the relation between descriptional complexity and algorithmic probability , 1981, 22nd Annual Symposium on Foundations of Computer Science (sfcs 1981).

[47]  Allen H. Brady.  The determination of the value of Rado’s noncomputable function Σ() for four-state Turing machines , 1983 .

[48]  Arto Salomaa,et al.  Computation and Automata , 1984 .

[49]  Derick Wood,et al.  Theory of computation , 1986 .

[50]  Gregory J. Chaitin,et al.  Computing the Busy Beaver Function , 1987 .

[51]  D. Ackley.  A connectionist machine for genetic hillclimbing , 1987 .

[52]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[53]  Allen H. Brady.  The busy beaver game and the meaning of life , 1988 .

[54]  Arnold Oberschelp,et al.  Castor quadruplorum , 1988, Arch. Math. Log..

[55]  Daniel L. Stein,et al.  Lectures In The Sciences Of Complexity , 1989 .

[56]  Heiner Marxen,et al.  Attacking the Busy Beaver 5 , 1990, Bull. EATCS.

[57]  Quentin F. Stout,et al.  The complex behavior of simple machines , 1990 .

[58]  Stewart W. Wilson.  GA-Easy Does Not Imply Steepest-Ascent Optimizable , 1991, ICGA.

[59]  Lawrence Davis,et al.  Bit-Climbing, Representational Bias, and Test Suite Design , 1991, ICGA.

[60]  J. C. Martin,et al.  Introduction to Languages and the Theory of Computation" 3rd Ed , 1991 .

[61]  Jon Barwise,et al.  Turing's World 3.0 : an introduction to computability theory : for the Macintosh , 1991 .

[62]  Bryant A. Julstrom.  A bound on the shift function in terms of the Busy Beaver function , 1992, SIGA.

[63]  Rolf Herken,et al.  The Universal Turing Machine: A Half-Century Survey , 1992 .

[64]  Pascal Michel,et al.  Busy beaver competition and Collatz-like problems , 1993, Arch. Math. Log..

[65]  HE Ixtroductiont,et al.  The Bell System Technical Journal , 2022 .