Context-Driven Predictions

Markov models have been a keystone in Artificial Intelligence for many decades. However, they remain unsatisfactory when the environment modelled is partially observable. There are pathological examples where no history of fixed length is sufficient for accurate prediction or decision making. On the other hand, working with a hidden state (like in Hidden Markov Models or Partially Observable Markov Decision Processes) has a high computational cost. In order to circumvent this problem, we suggest the use of a context-based model. Our approach replaces strict transition probabilities by influences on transitions. The method proposed provides a trade-off between a fully and partially observable model. We also discuss the capacity of our framework to model hierarchical knowledge and abstraction. Simple examples are given in order to show the advantages of the algorithm.

[1]  B. Schölkopf,et al.  Modeling Human Motion Using Binary Latent Variables , 2007 .

[2]  Richard S. Sutton,et al.  Temporal-Difference Networks , 2004, NIPS.

[3]  Geoffrey E. Hinton,et al.  Massively Parallel Architectures for AI: NETL, Thistle, and Boltzmann Machines , 1983, AAAI.

[4]  Andrew McCallum,et al.  Reinforcement learning with selective perception and hidden state , 1996 .

[5]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[6]  Richard S. Sutton,et al.  TD Models: Modeling the World at a Mixture of Time Scales , 1995, ICML.

[7]  Doina Precup,et al.  Sparse Distributed Memories for On-Line Value-Based Reinforcement Learning , 2004, ECML.

[8]  Richard S. Sutton,et al.  Associative search network: A reinforcement learning associative memory , 1981, Biological Cybernetics.

[9]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[10]  Pentti Kanerva,et al.  Sparse distributed memory and related models , 1993 .

[11]  Nils J. Nilsson,et al.  Artificial Intelligence , 1974, IFIP Congress.

[12]  G. Miller,et al.  Cognitive science. , 1981, Science.

[13]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[14]  R. Lathe Phd by thesis , 1988, Nature.

[15]  M. V. Rossum,et al.  In Neural Computation , 2022 .

[16]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[17]  Richard S. Sutton,et al.  Predictive Representations of State , 2001, NIPS.

[18]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[19]  S. Griffis EDITOR , 1997, Journal of Navigation.

[20]  Steve Temple,et al.  Sparse distributed memory using N-of-M codes , 2004, Neural Networks.

[21]  Thomas G. Dietterich,et al.  In Advances in Neural Information Processing Systems 12 , 1991, NIPS 1991.

[22]  Diane J. Cook,et al.  Online Sequential Prediction via Incremental Parsing: The Active LeZi Algorithm , 2007, IEEE Intelligent Systems.

[23]  Thomas G. Dietterich What is machine learning? , 2020, Archives of Disease in Childhood.

[24]  Pattie Maes,et al.  Learning Navigational Behaviors using a Predictive Sparse Distributed Memory , 1996 .

[25]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[26]  Joy Bose,et al.  An associative memory for the on-line recognition and prediction of temporal sequences , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..