Incremental 2-directional 2-dimensional linear discriminant analysis for multitask pattern recognition

In this paper, we propose an incremental 2-directional 2-dimensional linear discriminant analysis (I-(2D)2LDA) for multitask pattern recognition (MTPR) problems in which a chunk of training data for a particular task are given sequentially and the task is switched to another related task one after another. In I-(2D)2LDA, a discriminant space of the current task spanned by 2 types of discriminant vectors is augmented with effective discriminant vectors that are selected from other tasks based on the class separability. We call the selective augmentation of discriminant vectors knowledge transfer of feature space. In the experiments, the proposed I-(2D)2LDA is evaluated for the three tasks using the ORL face data set: person identification (Task 1), gender recognition (Task 2), and young-senior discrimination (Task 3). The results show that the knowledge transfer works well for Tasks 2 and 3; that is, the test performance of gender recognition and that of young-senior discrimination are enhanced.

[1]  Shaoning Pang,et al.  Incremental linear discriminant analysis for classification of data streams , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[2]  Sebastian Thrun,et al.  Learning to Learn , 1998, Springer US.

[3]  D. Silver,et al.  Selective Functional Transfer : Inductive Bias from Related Tasks , 2001 .

[4]  John C. Platt.  A Resource-Allocating Network for Function Interpolation , 1991, Neural Computation.

[5]  Yaser S. Abu-Mostafa,et al.  Learning from hints in neural networks , 1990, J. Complex..

[6]  Rich Caruana,et al.  Multitask Learning , 1997, Machine-mediated learning.

[7]  Rich Caruana,et al.  Learning Many Related Tasks at the Same Time with Backpropagation , 1994, NIPS.

[8]  Gert Cauwenberghs,et al.  Incremental and Decremental Support Vector Machine Learning , 2000, NIPS.

[9]  Seiichi Ozawa,et al.  A Multitask Learning Model for Online Pattern Recognition , 2009, IEEE Transactions on Neural Networks.

[10]  Shigeo Abe,et al.  An Efficient Incremental Kernel Principal Component Analysis for Online Feature Selection , 2007, 2007 International Joint Conference on Neural Networks.

[11]  Yoshua Bengio,et al.  Bias learning, knowledge sharing , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[12]  Stephen Grossberg,et al.  The ART of adaptive pattern recognition by a self-organizing neural network , 1988, Computer.

[13]  Massimiliano Pontil,et al.  Multi-Task Feature Learning , 2006, NIPS.

[14]  Nikola K. Kasabov,et al.  Incremental linear discriminant analysis for evolving feature spaces in multitask pattern recognition problems , 2010, Evol. Syst..

[15]  Shigeo Abe,et al.  Incremental learning of feature space and classifier for face recognition , 2005, Neural Networks.

[16]  Nikola Kasabov,et al.  Evolving Connectionist Systems: Methods and Applications in Bioinformatics, Brain Study and Intelligent Machines , 2002, IEEE Transactions on Neural Networks.

[17]  Palaiahnakote Shivakumara,et al.  (2D)2LDA: An efficient approach for face recognition , 2006, Pattern Recognit..