A Monte Carlo Algorithm for Universally Optimal Bayesian Sequence Prediction and Planning

The aim of this work is to address the question of whether we can in principle design rational decision-making agents or artificial intelligences embedded in computable physics such that their decisions are optimal in reasonable mathematical senses. Recent developments in rare event probability estimation, recursive bayesian inference, neural networks, and probabilistic planning are sufficient to explicitly approximate reinforcement learners of the AIXI style with non-trivial model classes (here, the class of resource-bounded Turing machines). Consideration of the effects of resource limitations in a concrete implementation leads to insights about possible architectures for learning systems using optimal decision makers as components.

[1]  Jürgen Schmidhuber,et al.  Learning Complex, Extended Sequences Using the Principle of History Compression , 1992, Neural Computation.

[2]  Jürgen Schmidhuber,et al.  Kalman filters improve LSTM network performance in problems unsolvable by traditional recurrent nets , 2003, Neural Networks.

[3]  Yishay Mansour,et al.  A Sparse Sampling Algorithm for Near-Optimal Planning in Large Markov Decision Processes , 1999, Machine Learning.

[4]  Heikki Hyotyniemi,et al.  Turing Machines Are Recurrent Neural Networks , 1996 .

[5]  Nando de Freitas,et al.  Reversible Jump MCMC Simulated Annealing for Neural Networks , 2000, UAI.

[6]  Ofi rNw8x'pyzm,et al.  The Speed Prior: A New Simplicity Measure Yielding Near-Optimal Computable Predictions , 2002 .

[7]  Giorgi Japaridze,et al.  Introduction to computability logic , 2003, Ann. Pure Appl. Log..

[8]  John R. Boyd,et al.  The Essence of Winning and Losing , 2012 .

[9]  Jürgen Schmidhuber,et al.  Optimal Ordered Problem Solver , 2002, Machine Learning.

[10]  Jürgen Schmidhuber,et al.  Ultimate Cognition à la Gödel , 2009, Cognitive Computation.

[11]  Hava T. Siegelmann,et al.  Neural networks and analog computation - beyond the Turing limit , 1999, Progress in theoretical computer science.

[12]  D. Madigan,et al.  A one-pass sequential Monte Carlo method for Bayesian analysis of massive datasets , 2006 .

[13]  Jürgen Schmidhuber,et al.  Flat Minima , 1997, Neural Computation.

[14]  Marcus Hutter,et al.  Universal Artificial Intellegence - Sequential Decisions Based on Algorithmic Probability , 2005, Texts in Theoretical Computer Science. An EATCS Series.

[15]  Dirk P. Kroese,et al.  An Efficient Algorithm for Rare-event Probability Estimation, Combinatorial Optimization, and Counting , 2008 .

[16]  M. V. Rossum,et al.  In Neural Computation , 2022 .