Anticipatory Behavior in Adaptive Learning Systems

The paper describes a new approach toward the study of anticipatory behavior in adaptive learning systems, an approach based on electrophysiological evidence of anticipatory behavior of the human brain. The basic idea is to study brain potentials related to anticipation of some event. Here we describe our work with the CNV anticipatory potential. We extended the classical CNV paradigm introducing a brain-computer interface that allows observation of the adaptive change of the cognitive environment of the subject. We obtained a cognitive phenomenon which generates a trace which we denoted as Electroexpectogram (EXG). It shows a learning process represented by the learned anticipation. Emotional Petri Nets are used in explanation of the EXG paradigm in terms of consequence driven systems theory.

[1]  Francis Crick,et al.  REM sleep and neural nets , 1995, Behavioural Brain Research.

[2]  Carlyle T. Smith,et al.  Sleep states and memory processes , 1995, Behavioural Brain Research.

[3]  Naftali Tishby,et al.  The information bottleneck method , 2000, ArXiv.

[4]  Jean-Gabriel Ganascia,et al.  Learning Strategies in Games by Anticipation , 1997, IJCAI.

[5]  Andrew G. Barto,et al.  Learning to Act Using Real-Time Dynamic Programming , 1995, Artif. Intell..

[6]  T. Ziemke,et al.  Making sense of embodiment : simulation theories and the sharing of neural circuitry , 2004 .

[7]  H. Maturana,et al.  Autopoiesis and Cognition : The Realization of the Living (Boston Studies in the Philosophy of Scie , 1980 .

[8]  H. Pashler.  The Psychology of Attention , 1997 .

[9]  Norio Konno,et al.  Artificial life with autonomously emerging boundaries , 1991 .

[10]  Andreas Herzig,et al.  A modal logic for epistemic tests , 2000, ECAI.

[11]  George Dimitri Konidaris,et al.  An Architecture for Behavior-Based Reinforcement Learning , 2005, Adapt. Behav..

[12]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[13]  Maja J. Mataric,et al.  Integration of representation into goal-driven behavior-based robots , 1992, IEEE Trans. Robotics Autom..

[14]  H A Simon,et al.  The theory of learning by doing. , 1979, Psychological review.

[15]  I. Tsuda.  Toward an interpretation of dynamic neural activity in terms of chaotic dynamical systems. , 2001, The Behavioral and brain sciences.

[16]  A. Noë,et al.  A sensorimotor account of vision and visual consciousness. , 2001, The Behavioral and brain sciences.

[17]  John E. Laird,et al.  It knows what you're going to do: adding anticipation to a Quakebot , 2001, AGENTS '01.

[18]  Stewart W. Wilson.  Function approximation with a classifier system , 2001 .

[19]  Olivier Sigaud,et al.  Combining latent learning with dynamic programming in the modular anticipatory classifier system , 2005, Eur. J. Oper. Res..

[20]  G. Hesslow.  Conscious thought as simulation of behaviour and perception , 2002, Trends in Cognitive Sciences.

[21]  Jun Tani,et al.  On the dynamics of robot exploration learning , 2002, Cognitive Systems Research.

[22]  Stefano Nolfi,et al.  How Co-Evolution can Enhance the Adaptive Power of Artificial Evolution: Implications for Evolutionary Robotics , 1998, EvoRobot.

[23]  Martin V. Butz,et al.  Anticipatory Behavior: Exploiting Knowledge About the Future to Improve Current Behavior , 2003, ABiALS.

[24]  J. Born,et al.  Sleep inspires insight , 2004, Nature.

[25]  Paul Davidsson,et al.  A Framework for Organization and Representation of Concept Knowledge in Autonomous Agents , 1993, SCAI.

[26]  Richard E. Korf,et al.  Real-Time Heuristic Search , 1990, Artif. Intell..

[27]  G. Cullity,et al.  The architecture of reason : the structure and substance of rationality , 2003 .

[28]  Richard S. Sutton,et al.  Dyna, an integrated architecture for learning, planning, and reacting , 1990, SGAR.

[29]  Carlyle Smith,et al.  Prolonged increases in both PS and number of REMs following a shuttle avoidance task , 1986, Physiology & Behavior.

[30]  Marvin Minsky,et al.  A framework for representing knowledge , 1974 .

[31]  Jun Tani,et al.  Model-based learning for mobile robot navigation from the dynamical systems perspective , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[32]  Inman Harvey,et al.  Is There Another New Factor in Evolution? , 1996, Evolutionary Computation.

[33]  Wolfgang Stolzmann,et al.  An Introduction to Anticipatory Classifier Systems , 1999, Learning Classifier Systems.

[34]  D. Davidson.  Essays on actions and events , 1980 .

[35]  Stanley J. Rosenschein,et al.  A dynamical systems perspective on agent-environment interaction , 1996 .

[36]  Jeffrey L. Elman,et al.  Learning and Evolution in Neural Networks , 1994, Adapt. Behav..

[37]  Germund Hesslow,et al.  EXPLORING INTERNAL SIMULATION OF PERCEPTION IN MOBILE ROBOTS , 2001 .

[38]  Robert Stickgold,et al.  To dream or not to dream? Relevant data from new neuroimaging and electrophysiological studies , 1998, Current Opinion in Neurobiology.

[39]  M. Wilson,et al.  Temporally Structured Replay of Awake Hippocampal Ensemble Activity during Rapid Eye Movement Sleep , 2001, Neuron.

[40]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[41]  Hector J. Levesque,et al.  Intention is Choice with Commitment , 1990, Artif. Intell..

[42]  Olivier Sigaud,et al.  YACS: Combining Dynamic Programming with Generalization in Classifier Systems , 2000, IWLCS.

[43]  H. Pashler,et al.  The Psychology of Attention , 2000 .

[44]  Stefano Nolfi,et al.  Learning to Adapt to Changing Environments in Evolving Neural Networks , 1996, Adapt. Behav..

[45]  Richard S. Sutton,et al.  Temporal credit assignment in reinforcement learning , 1984 .

[46]  Cristiano Castelfranchi,et al.  Reasons: belief support and goal dynamics , 1996 .

[47]  Richard S. Sutton,et al.  Reinforcement learning architectures for animats , 1991 .

[48]  Stefano Nolfi.  How Learning and Evolution Interact: The Case of a Learning Task which Differs from the Evolutionary Task , 1999, Adapt. Behav..

[49]  Jun Tani,et al.  Self-Organization of Behavior Primitives as Multiple Attractor Dynamics by the "Forwarding Forward Model" network , 2002 .

[50]  Peter Stone,et al.  Anticipation as a key for collaboration in a team of agents: a case study in robotic soccer , 1999, Optics East.

[51]  A. Martin V. Butz,et al.  The anticipatory classifier system and genetic generalization , 2002, Natural Computing.

[52]  Jun Tani,et al.  Learning to generate articulated behavior through the bottom-up and the top-down interaction processes , 2003, Neural Networks.

[53]  Naohiro Fukumura,et al.  Embedding a grammatical description in deterministic chaos: an experiment in recurrent neural learning , 1995, Biological Cybernetics.

[54]  John F. Kolen,et al.  Exploring the computational capabilities of recurrent neural networks , 1995 .

[55]  Martin V. Butz,et al.  Internal Models and Anticipations in Adaptive Learning Systems , 2003, ABiALS.

[56]  Aude Billard,et al.  Estimating Future Reward in Reinforcement Learning Animats using Associative Learning , 2004 .

[57]  Young,et al.  Inferring statistical complexity. , 1989, Physical review letters.

[58]  Ronald C. Arkin,et al.  Anticipatory robot navigation by simultaneously localizing and building a cognitive map , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[59]  Michael I. Jordan,et al.  Forward Models: Supervised Learning with a Distal Teacher , 1992, Cogn. Sci..

[60]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[61]  Anand S. Rao,et al.  An Abstract Architecture for Rational Agents , 1992, KR.

[62]  John-Jules Ch. Meyer,et al.  Tests as Epistemic Updates , 1994, ECAI.

[63]  Jun Tani,et al.  An Interpretation of the "Self" From the Dynamical Systems Perspective: A Constructivist Approach , 1998 .

[64]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[65]  Stefano Nolfi,et al.  Learning to perceive the world as articulated: an approach for hierarchical learning in sensory-motor systems , 1998, Neural Networks.

[66]  J. Pollack.  The Induction of Dynamical Recognizers , 1996, Machine Learning.