Gödel Machines: Fully Self-referential Optimal Universal Self-improvers

We present the first class of mathematically rigorous, general, fully self-referential, self-improving, optimally efficient problem solvers. Inspired by Kurt Godel’s celebrated self-referential formulas (1931), such a problem solver rewrites any part of its own code as soon as it has found a proof that the rewrite is useful, where the problem-dependent utility function and the hardware and the entire initial code are described by axioms encoded in an initial proof searcher which is also part of the initial code. The searcher systematically and efficiently tests computable proof techniques (programs whose outputs are proofs) until it finds a provably useful, computable self-rewrite. We show that such a self-rewrite is globally optimal—no local maxima!—since the code first had to prove that it is not useful to continue the proof search for alternative self-rewrites. Unlike previous non-self-referential methods based on hardwired proof searchers, ours not only boasts an optimal order of complexity but can optimally reduce any slowdowns hidden by the O()-notation, provided the utility of such speed-ups is provable at all.

[1]  Leopold Löwenheim Über Möglichkeiten im Relativkalkül , 1915 .

[2]  W. Heisenberg A quantum-theoretical reinterpretation of kinematic and mechanical relations , 1925 .

[3]  W. Heisenberg Über den anschaulichen Inhalt der quantentheoretischen Kinematik und Mechanik , 1927 .

[4]  K. Gödel Über formal unentscheidbare Sätze der Principia Mathematica und verwandter Systeme I , 1931 .

[5]  A. Kolmogoroff Grundbegriffe der Wahrscheinlichkeitsrechnung , 1933 .

[6]  A. Turing On computable numbers, with an application to the Entscheidungsproblem , 1937, Proc. London Math. Soc..

[7]  Arthur L. Samuel,et al.  Some Studies in Machine Learning Using the Game of Checkers , 1967, IBM J. Res. Dev..

[8]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part I , 1964, Inf. Control..

[9]  R. Bellman,et al.  V. Adaptive Control Processes , 1964 .

[10]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part II , 1964, Inf. Control..

[11]  Manuel Blum,et al.  A Machine-Independent Theory of the Complexity of Recursive Functions , 1967, JACM.

[12]  A. Kolmogorov Three approaches to the quantitative definition of information , 1968 .

[13]  Charles H. Moore,et al.  Forth - a language for interactive computing , 1970 .

[14]  Manuel Blum,et al.  On Effective Procedures for Speeding Up Algorithms , 1971, JACM.

[15]  G. Chaitin A Theory of Program Size Formally Identical to Information Theory , 1975, JACM.

[16]  Ray J. Solomonoff,et al.  Complexity-based induction systems: Comparisons and convergence theorems , 1978, IEEE Trans. Inf. Theory.

[17]  D. Hofstadter,et al.  Godel, Escher, Bach: An Eternal Golden Braid , 1979 .

[18]  Douglas B. Lenat,et al.  Theory Formation by Heuristic Search , 1983, Artificial Intelligence.

[19]  H. Cantor Ueber eine Eigenschaft des Inbegriffs aller reellen algebraischen Zahlen. , 1984 .

[20]  Leonid A. Levin,et al.  Randomness Conservation Inequalities; Information and Independence in Mathematical Theories , 1984, Inf. Control..

[21]  Nichael Lynn Cramer,et al.  A Representation for the Adaptive Generation of Simple Sequential Programs , 1985, ICGA.

[22]  John H. Holland,et al.  Properties of the Bucket Brigade , 1985, ICGA.

[23]  William F. Clocksin,et al.  Programming in Prolog , 1987, Springer Berlin Heidelberg.

[24]  Jürgen Schmidhuber,et al.  Reinforcement Learning in Markovian and Non-Markovian Environments , 1990, NIPS.

[25]  Toshio Odanaka,et al.  ADAPTIVE CONTROL PROCESSES , 1990 .

[26]  Konrad Zuse,et al.  Rechnender Raum , 1991, Physik und Informatik.

[27]  Jürgen Schmidhuber,et al.  A ‘Self-Referential’ Weight Matrix , 1993 .

[28]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 2019, Texts in Computer Science.

[29]  R. Penrose,et al.  Shadows of the Mind , 1994 .

[30]  Juergen Schmidhuber,et al.  On learning how to learn learning strategies , 1994 .

[31]  Corso Elvezia Discovering Solutions with Low Kolmogorov Complexity and High Generalization Capability , 1995 .

[32]  D. Wolpert,et al.  No Free Lunch Theorems for Search , 1995 .

[33]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[34]  Melvin Fitting,et al.  First-Order Logic and Automated Theorem Proving , 1990, Graduate Texts in Computer Science.

[35]  Jieyu Zhao,et al.  Simple Principles of Metalearning , 1996 .

[36]  Wolfgang Banzhaf,et al.  Genetic Programming: An Introduction , 1997 .

[37]  Jürgen Schmidhuber,et al.  Discovering Neural Nets with Low Kolmogorov Complexity and High Generalization Capability , 1997, Neural Networks.

[38]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[39]  Jürgen Schmidhuber,et al.  A Computer Scientist's View of Life, the Universe, and Everything , 1999, Foundations of Computer Science: Potential - Theory - Cognition.

[40]  Wilfried Brauer,et al.  Foundations of computer science : potential--theory--cognition , 1997 .

[41]  K. Popper All life is problem solving , 1997 .

[42]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[43]  Jürgen Schmidhuber,et al.  Reinforcement Learning with Self-Modifying Policies , 1998, Learning to Learn.

[44]  Sebastian Thrun,et al.  Learning to Learn , 1998, Springer US.

[45]  C. Koch,et al.  Consciousness and neuroscience. , 1998, Cerebral cortex.

[46]  Jürgen Schmidhuber,et al.  Algorithmic Theories of Everything , 2000, ArXiv.

[47]  Sepp Hochreiter,et al.  Learning to Learn Using Gradient Descent , 2001, ICANN.

[48]  Marcus Hutter,et al.  Towards a Universal Theory of Artificial Intelligence Based on Algorithmic Probability and Sequential Decisions , 2000, ECML.

[49]  Jürgen Schmidhuber,et al.  The Speed Prior: A New Simplicity Measure Yielding Near-Optimal Computable Predictions , 2002, COLT.

[50]  Ofi rNw8x'pyzm,et al.  The Speed Prior: A New Simplicity Measure Yielding Near-Optimal Computable Predictions , 2002 .

[51]  Jürgen Schmidhuber,et al.  Hierarchies of Generalized Kolmogorov Complexities and Nonenumerable Universal Measures Computable in the Limit , 2002, Int. J. Found. Comput. Sci..

[52]  Jürgen Schmidhuber,et al.  Bias-Optimal Incremental Problem Solving , 2002, NIPS.

[53]  Marcus Hutter The Fastest and Shortest Algorithm for all Well-Defined Problems , 2002, Int. J. Found. Comput. Sci..

[54]  Marcus Hutter,et al.  Self-Optimizing and Pareto-Optimal Policies in General Environments based on Bayes-Mixtures , 2002, COLT.

[55]  Jürgen Schmidhuber,et al.  Goedel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements , 2003, ArXiv.

[56]  Jürgen Schmidhuber,et al.  Optimal Ordered Problem Solver , 2002, Machine Learning.

[57]  Jürgen Schmidhuber,et al.  Shifting Inductive Bias with Success-Story Algorithm, Adaptive Levin Search, and Incremental Self-Improvement , 1997, Machine Learning.

[58]  Jürgen Schmidhuber,et al.  Gödel Machines: Towards a Technical Justification of Consciousness , 2005, Adaptive Agents and Multi-Agent Systems.

[59]  Daniel Kudenko,et al.  Adaptive Agents and Multi-Agent Systems II: Adaptation and Multi-Agent Learning , 2003, Adaptive Agents and Multi-Agent Systems.

[60]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[61]  Jürgen Schmidhuber,et al.  Completely Self-referential Optimal Reinforcement Learners , 2005, ICANN.

[62]  Jürgen Schmidhuber,et al.  The New AI: General & Sound & Relevant for Physics , 2003, Artificial General Intelligence.