Sequence learning: from recognition and prediction to sequential decision making

2 1094-7167/01/$10.00 © 2001 IEEE IEEE INTELLIGENT SYSTEMS So, it’s logical that sequence learning is an important component of learning in many task domains of intelligent systems: inference, planning, reasoning, robotics, natural language processing, speech recognition, adaptive control, time series prediction, financial engineering, DNA sequencing, and so on. Naturally, the unique perspectives of these domains lead to different sequence-learning approaches. These approaches deal with somewhat differently formulated sequence-learning problems (for example, some with actions and some without) and with different aspects of sequence learning (for example, sequence prediction versus sequence recognition). Despite the plethora of approaches, sequence learning is still difficult. We believe that the right approach to improving sequence learning is to first better understand the state of the art in the different disciplines related to this topic. This requires comparing, contrasting, and combining the existing techniques, approaches, and paradigms, to develop better, more powerful algorithms. Toward that end, we present here a brief tutorial on sequence learning.

[1]  Ron Sun,et al.  Autonomous learning of sequential tasks: experiments and analyses , 1998, IEEE Trans. Neural Networks.

[2]  Craig Boutilier,et al.  Abstraction and Approximate Decision-Theoretic Planning , 1997, Artif. Intell..

[3]  Andrew McCallum,et al.  Learning to Use Selective Attention and Short-Term Memory in Sequential Tasks , 1996 .

[4]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[5]  Chen K. Tham,et al.  Reinforcement learning of multiple tasks using a hierarchical CMAC architecture , 1995, Robotics Auton. Syst..

[6]  Ron Sun,et al.  Multi-agent reinforcement learning: weighting and partitioning , 1999, Neural Networks.

[7]  Ron Sun,et al.  Self-segmentation of sequences: automatic formation of hierarchies of sequential behaviors , 2000, IEEE Trans. Syst. Man Cybern. Part B.

[8]  TesauroGerald.  Practical Issues in Temporal Difference Learning , 1992 .

[9]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[10]  Richard S. Sutton,et al.  Reinforcement Learning , 1992, Handbook of Machine Learning.

[11]  Ron Sun,et al.  Learning to plan probabilistically from neural networks , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[12]  Ron Sun,et al.  Learning Plans without a priori Knowledge , 2000, Adapt. Behav..

[13]  Lipo Wang,et al.  Artificial neural networks - oscillations, chaos, and sequence processing , 1993 .

[14]  H Sompolinsky,et al.  Associative neural network model for the generation of temporal patterns. Theory and application to central pattern generators. , 1988, Biophysical journal.

[15]  P. Andersen,et al.  [Learning and memory]. , 1995, Tidsskrift for den Norske laegeforening : tidsskrift for praktisk medicin, ny raekke.

[16]  Ian H. Witten,et al.  Identifying Hierarchical Structure in Sequences: A linear-time algorithm , 1997, J. Artif. Intell. Res..

[17]  Personnaz,et al.  Storage and retrieval of complex sequences in neural networks. , 1988, Physical review. A, General physics.

[18]  Ron Sun,et al.  From implicit skills to explicit knowledge: a bottom-up model of skill learning , 2001, Cogn. Sci..

[19]  Marco Gori,et al.  Adaptive Processing of Sequences and Data Structures , 1998, Lecture Notes in Computer Science.

[20]  John J. Grefenstette,et al.  The Evolution of Strategies for Multiagent Environments , 1992, Adapt. Behav..

[21]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[22]  Lisa Meeden,et al.  An incremental approach to developing intelligent neural network controllers for robots , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[23]  David Chapman,et al.  What are plans for? , 1990, Robotics Auton. Syst..

[24]  C. Lee Giles,et al.  Learning a class of large finite state machines with a recurrent neural network , 1995, Neural Networks.

[25]  Ron Sun,et al.  Self-segmentation of sequences , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[26]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[27]  T. SHALLICE,et al.  Learning and Memory , 1970, Nature.

[28]  Maja J. Matarić,et al.  Learning to Use Selective Attention and Short-Term Memory in Sequential Tasks , 1996 .

[29]  L. Baum,et al.  An inequality and associated maximization technique in statistical estimation of probabilistic functions of a Markov process , 1972 .

[30]  C. L. Giles,et al.  Sequence Learning - Paradigms, Algorithms, and Applications , 2001 .