Radial Basis Function Network for Multitask Pattern Recognition

In this paper, a novel type of radial basis function network is proposed for multitask pattern recognition. We assume that recognition tasks are switched sequentially without notice to a learner and they have relatedness to some extent. We further assume that training data are given to learn one by one and they are discarded after learning. To learn a recognition system incrementally in such a multitask environment, we propose Resource Allocating Network for Multi-Task Pattern Recognition (RAN-MTPR). There are five distinguished functions in RAN-MTPR: one-pass incremental learning, task change detection, task categorization, knowledge consolidation, and knowledge transfer. The first three functions enable RAN-MTPR not only to acquire and accumulate knowledge of tasks stably but also to allocate classes to appropriate tasks unless task labels are not explicitly given. The fourth function enables RAN-MTPR to recover the failure in task categorization by minimizing the conflict in class allocation to tasks. The fifth function, knowledge transfer from one task to another, is realized by sharing the internal representation of a hidden layer with different tasks and by transferring class information of the most related task to a new task. The experimental results show that the recognition performance of RAN-MTPR is enhanced by introducing the two types of knowledge transfer and the consolidation works well to reduce the failure in task change detection and task categorization if the RBF width is properly set.

[1]  Nikola Kasabov,et al.  Evolving Connectionist Systems: Methods and Applications in Bioinformatics, Brain Study and Intelligent Machines , 2002, IEEE Transactions on Neural Networks.

[2]  Rich Caruana,et al.  Multitask Learning , 1997, Machine-mediated learning.

[3]  Haibo He,et al.  Towards incremental learning of nonstationary imbalanced data stream: a multiple selectively recursive approach , 2011, Evol. Syst..

[4]  Shaoning Pang,et al.  Incremental Learning of Chunk Data for Online Pattern Classification Systems , 2008, IEEE Transactions on Neural Networks.

[5]  Stephen Grossberg,et al.  The ART of adaptive pattern recognition by a self-organizing neural network , 1988, Computer.

[6]  David Taniar,et al.  Computational Science and Its Applications - ICCSA 2005, International Conference, Singapore, May 9-12, 2005, Proceedings, Part I , 2005, ICCSA.

[7]  Yoshua Bengio,et al.  Bias learning, knowledge sharing , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[8]  Minho Lee,et al.  Autonomous Incremental Visual Environment Perception Based on Visual Selective Attention , 2007, 2007 International Joint Conference on Neural Networks.

[9]  Rich Caruana,et al.  Multitask pattern recognition for autonomous robots , 1998, Proceedings. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. Innovations in Theory, Practice and Applications (Cat. No.98CH36190).

[10]  John C. Platt.  A Resource-Allocating Network for Function Interpolation , 1991, Neural Computation.

[11]  Lorien Y. Pratt,et al.  A Survey of Connectionist Network Reuse Through Transfer , 1998, Learning to Learn.

[12]  Seiichi Ozawa,et al.  Incremental learning in dynamic environments using neural network with long-term memory , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[13]  Kapil Keshao Wankhade,et al.  A fast and light classifier for data streams , 2010, Evol. Syst..

[14]  Seiichi Ozawa,et al.  A Multitask Learning Model for Online Pattern Recognition , 2009, IEEE Transactions on Neural Networks.

[15]  Plamen P. Angelov,et al.  Evolving classification of agents’ behaviors: a general approach , 2010, Evol. Syst..

[16]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[17]  Jonathan Baxter,et al.  A Bayesian/Information Theoretic Model of Learning to Learn via Multiple Task Sampling , 1997, Machine Learning.

[18]  Naohiro Ishii,et al.  Incremental learning methods with retrieving of interfered patterns , 1999, IEEE Trans. Neural Networks.

[19]  Sebastian Thrun,et al.  Learning to Learn , 1998, Springer US.

[20]  D. Silver,et al.  Selective Functional Transfer : Inductive Bias from Related Tasks , 2001 .

[21]  Shiliang Sun,et al.  A Multitask Learning Approach to Face Recognition Based on Neural Networks , 2008, IDEAL.

[22]  Shigeo Abe,et al.  A fast incremental learning algorithm of RBF networks with long-term memory , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[23]  José de Jesús Rubio,et al.  Backpropagation to train an evolving radial basis function neural network , 2010, Evol. Syst..

[24]  Narasimhan Sundararajan,et al.  A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation , 2005, IEEE Transactions on Neural Networks.

[25]  Nikola K. Kasabov,et al.  Incremental linear discriminant analysis for evolving feature spaces in multitask pattern recognition problems , 2010, Evol. Syst..

[26]  Shigeo Abe,et al.  Incremental learning of feature space and classifier for face recognition , 2005, Neural Networks.

[27]  Shaoning Pang,et al.  Incremental linear discriminant analysis for classification of data streams , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).