Investigating Experience: Temporal Coherence and Empirical Knowledge Representation

This thesis investigates the idea of artificial intelligence as an agent making sense of its experience, illustrating some of the benefits of representing knowledge as predictions of future experience. Experience is here defined as the temporal sequence of sensations and actions that are the inputs and outputs of the agent. One characteristic of this sequence is that it can have temporal coherence: what is experienced in a short period of time is likely to be consistent. The first part of this thesis examines how an agent with dynamic memory can take advantage of the temporal coherence of its experience. Results in a simple prediction task and the more complex problem of Computer Go show how such an agent can dramatically improve on the performance of the best stationary solutions. The prediction task is then used to illustrate how temporal coherence can provide a natural testbed for meta-learning. In the second part of the thesis, the frameworks of predictive representations and options are adapted for use in knowledge representation. The traditional approach to knowledge representation for artificial intelligence uses the framework of formal logic, in which knowledge is dissociated from experience. The knowledge representation presented here is defined in terms of experience, predictions and time. This kind of representation is defined in this thesis as an empirical knowledge representation. Using objects as a case study, the final chapter shows how an empirical knowledge representation makes it possible to represent even abstract concepts in terms of experience.

[1]  L. Frank.  The Society for Research in Child Development , 1935 .

[2]  J. Piaget.  The construction of reality in the child , 1954 .

[3]  W. Bean.  Personal Knowledge: Towards a Post-Critical Philosophy , 1961 .

[4]  Herbert P. Ginsburg,et al.  Piaget's Theory of Intellectual Development: An Introduction , 1969 .

[5]  T. Bower.  A primer of infant development , 1977 .

[6]  Avron Barr,et al.  The Handbook of Artificial Intelligence, Volume 1 , 1982 .

[7]  H. Furth.  Object permanence in five-month-old infants. , 1987, Cognition.

[8]  Elizabeth S. Spelke,et al.  Principles of Object Perception , 1990, Cogn. Sci..

[9]  Ramanathan V. Guha,et al.  Cyc: toward programs with common sense , 1990, CACM.

[10]  John R. Searle,et al.  Minds, brains, and programs , 1980, Behavioral and Brain Sciences.

[11]  E. Spelke.  Physical knowledge in infancy : Reflections on Piaget's theory , 1991 .

[12]  Anders Krogh,et al.  A Simple Weight Decay Can Improve Generalization , 1991, NIPS.

[13]  R. A. Brooks,et al.  Intelligence without Representation , 1991, Artif. Intell..

[14]  Richard S. Sutton,et al.  Adapting Bias by Gradient Descent: An Incremental Version of Delta-Bar-Delta , 1992, AAAI.

[15]  E. Spelke,et al.  Perceiving and reasoning about objects: Insights from infants , 1993 .

[16]  E. Spelke.  Initial knowledge: six suggestions , 1994, Cognition.

[17]  J. J. Higgins,et al.  Concepts in Probability and Stochastic Modeling , 1994 .

[18]  Gerald Tesauro,et al.  Temporal Difference Learning and TD-Gammon , 1995, J. Int. Comput. Games Assoc..

[19]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[20]  Douglas B. Lenat,et al.  CYC: a large-scale investment in knowledge infrastructure , 1995, CACM.

[21]  J. Mark.  Introduction to radial basis function networks , 1996 .

[22]  John N. Tsitsiklis,et al.  Analysis of Temporal-Diffference Learning with Function Approximation , 1996, NIPS.

[23]  Maja J. Matari,et al.  Behavior-based Control: Examples from Navigation, Learning, and Group Behavior , 1997 .

[24]  Randy Goebel,et al.  Computational intelligence - a logical approach , 1998 .

[25]  Doina Precup,et al.  Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning , 1999, Artif. Intell..

[26]  Nicol N. Schraudolph,et al.  Local Gain Adaptation in Stochastic Gradient Descent , 1999 .

[27]  Doina Precup,et al.  Temporal abstraction in reinforcement learning , 2000, ICML 2000.

[28]  Ned Markosian,et al.  Physical Objects , 1959, Philosophy.

[29]  Richard S. Sutton,et al.  Predictive Representations of State , 2001, NIPS.

[30]  Terrence J. Sejnowski,et al.  Learning to evaluate Go positions via temporal difference methods , 2001 .

[31]  Andrew G. Barto,et al.  Autonomous discovery of temporal abstractions from interaction with an environment , 2002 .

[32]  Martin Müller,et al.  Computer Go , 2002, Artif. Intell..

[33]  J. van Leeuwen,et al.  Neural Networks: Tricks of the Trade , 2002, Lecture Notes in Computer Science.

[34]  Margaret Wilson,et al.  Six views of embodied cognition , 2002, Psychonomic bulletin & review.

[35]  H. Jaap van den Herik,et al.  Solving Go on Small Boards , 2003, J. Int. Comput. Games Assoc..

[36]  Michael L. Anderson.  Embodied Cognition: A field guide , 2003, Artif. Intell..

[37]  Nuttapong Chentanez,et al.  Intrinsically Motivated Reinforcement Learning , 2004, NIPS.

[38]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[39]  Michael R. James,et al.  Predictive State Representations: A New Theory for Modeling Dynamical Systems , 2004, UAI.

[40]  Gerald Tesauro,et al.  Practical issues in temporal difference learning , 1992, Machine Learning.

[41]  Guy Shani,et al.  A Survey of Model-Based and Model-Free Methods for Resolving Perceptual Aliasing , 2004 .

[42]  Richard S. Sutton,et al.  Temporal-Difference Networks , 2004, NIPS.

[43]  Mariarosaria Taddeo,et al.  Solving the symbol grounding problem: a critical review of fifteen years of research , 2005, J. Exp. Theor. Artif. Intell..

[44]  Richard S. Sutton,et al.  Temporal-Difference Networks with History , 2005, IJCAI.

[45]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[46]  Richard S. Sutton,et al.  Using Predictive Representations to Improve Generalization in Reinforcement Learning , 2005, IJCAI.

[47]  Richard S. Sutton,et al.  Temporal Abstraction in Temporal-difference Networks , 2005, NIPS.

[48]  Michael J. Witbrock,et al.  An Introduction to the Syntax and Content of Cyc , 2006, AAAI Spring Symposium: Formalizing and Compiling Background Knowledge and Its Applications to Knowledge Representation and Question Answering.

[49]  Michael J. Witbrock,et al.  Automated Population of Cyc: Extracting Information about Named-entities from the Web , 2006, FLAIRS.

[50]  R. Sutton.  Gain Adaptation Beats Least Squares , 2006 .

[51]  S. Harnad.  Symbol grounding problem , 1991, Scholarpedia.

[52]  Andrew G. Barto,et al.  Building Portable Options: Skill Transfer in Reinforcement Learning , 2007, IJCAI.

[53]  Vadim Bulitko,et al.  Grounding Abstractions in Predictive State Representations , 2007, IJCAI.

[54]  Vishal Soni,et al.  Relational Knowledge with Predictive State Representations , 2007, IJCAI.

[55]  Richard S. Sutton,et al.  On the role of tracking in stationary environments , 2007, ICML '07.

[56]  Michael J. Witbrock,et al.  Autonomous Classification of Knowledge into an Ontology , 2007, FLAIRS.

[57]  David Silver,et al.  Combining Online and Offline Learning in UCT , 2007 .

[58]  Richard S. Sutton,et al.  Reinforcement Learning of Local Shape in the Game of Go , 2007, IJCAI.