Incremental Linear Discriminant Analysis Using Sufficient Spanning Sets and Its Applications

This paper presents an incremental learning solution for Linear Discriminant Analysis (LDA) and its applications to object recognition problems. We apply the sufficient spanning set approximation in three steps i.e. update for the total scatter matrix, between-class scatter matrix and the projected data matrix, which leads an online solution which closely agrees with the batch solution in accuracy while significantly reducing the computational complexity. The algorithm yields an efficient solution to incremental LDA even when the number of classes as well as the set size is large. The incremental LDA method has been also shown useful for semi-supervised online learning. Label propagation is done by integrating the incremental LDA into an EM framework. The method has been demonstrated in the task of merging large datasets which were collected during MPEG standardization for face image retrieval, face authentication using the BANCA dataset, and object categorisation using the Caltech101 dataset.

[1]  Chi-Ho Chan.  Multi-scale local Binary Pattern Histogram for Face Recognition , 2007, ICB.

[2]  Luc Vandendorpe,et al.  Face Authentication Competition on the BANCA Database , 2004, ICBA.

[3]  Tomer Hertz,et al.  Learning a Mahalanobis Metric from Equivalence Constraints , 2005, J. Mach. Learn. Res..

[4]  Hui Xiong,et al.  IDR/QR: an incremental dimension reduction algorithm via QR decomposition , 2004, IEEE Transactions on Knowledge and Data Engineering.

[5]  Trevor Darrell,et al.  The pyramid match kernel: discriminative classification with sets of image features , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[6]  Horst Bischof,et al.  Incremental LDA Learning by Combining Reconstructive and Discriminative Approaches , 2007, BMVC.

[7]  Luc Vandendorpe,et al.  Face authentication test on the BANCA database , 2004, ICPR 2004.

[8]  Shaoning Pang,et al.  Incremental linear discriminant analysis for classification of data streams , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[9]  Hong Cheng,et al.  Sparsity induced similarity measure for label propagation , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[10]  Björn Stenger,et al.  AIDIA - Adaptive Interface for Display InterAction , 2008, BMVC.

[11]  Hyun-Chul Kim,et al.  Face recognition using LDA mixture model , 2002, Object recognition supported by user interaction for service robots.

[12]  Antonio Criminisi,et al.  Object categorization by learned universal visual dictionary , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[13]  Erkki Oja,et al.  Subspace methods of pattern recognition , 1983 .

[14]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[15]  Cordelia Schmid,et al.  Dimension Reduction and Classification Methods for Object Recognition in Vision , 2004 .

[16]  Xiaojin Zhu,et al.  Semi-Supervised Learning Literature Survey , 2005 .

[17]  Alex Holub,et al.  Exploiting Unlabelled Data for Hybrid Object Classification , 2005 .

[18]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[19]  Tat-Jun Chin,et al.  Incremental Kernel PCA for Efficient Non-linear Feature Extraction , 2006, BMVC.

[20]  Shuicheng Yan,et al.  An HOG-LBP human detector with partial occlusion handling , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[21]  Hiroshi Mizoguchi,et al.  Successive learning of linear discriminant analysis: Sanger-type algorithm , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[22]  Ralph R. Martin,et al.  Merging and Splitting Eigenspace Models , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  B. S. Manjunath,et al.  Introduction to MPEG-7: Multimedia Content Description Interface , 2002 .

[24]  B. V. K. Vijaya Kumar,et al.  Representational oriented component analysis (ROCA) for face recognition with one sample image per training class , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[25]  Juan Carlos Niebles,et al.  Unsupervised Learning of Human Action Categories Using Spatial-Temporal Words , 2006, BMVC.

[26]  Josef Kittler,et al.  Component-based LDA face description for image retrieval and MPEG-7 standardisation , 2005, Image Vis. Comput..

[27]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[28]  Hua Yu,et al.  A direct LDA algorithm for high-dimensional data - with application to face recognition , 2001, Pattern Recognit..

[29]  Bernt Schiele,et al.  Analyzing appearance and contour based methods for object categorization , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[30]  Pavel Pudil,et al.  Introduction to Statistical Pattern Recognition , 2006 .

[31]  Hua Li,et al.  IMMC: incremental maximum margin criterion , 2004, KDD.

[32]  Chi-Ho Chan,et al.  Face Video Competition , 2009, ICB.

[33]  Jieping Ye,et al.  Efficient Kernel Discriminant Analysis via QR Decomposition , 2004, NIPS.

[34]  Fei Wang,et al.  Label Propagation through Linear Neighborhoods , 2006, IEEE Transactions on Knowledge and Data Engineering.

[35]  Danijel Skocaj,et al.  Weighted and robust incremental method for subspace learning , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[36]  Jitendra Malik,et al.  Shape matching and object recognition using low distortion correspondences , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[37]  Josef Kittler,et al.  Incremental Linear Discriminant Analysis Using Sufficient Spanning Set Approximations , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[38]  Ying Wu,et al.  View-independent recognition of hand postures , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[39]  Ming-Hsuan Yang,et al.  Adaptive Discriminative Generative Model and Its Applications , 2004, NIPS.

[40]  Frédéric Jurie,et al.  Fast Discriminative Visual Codebooks using Randomized Clustering Forests , 2006, NIPS.

[41]  David G. Lowe,et al.  Multiclass Object Recognition with Sparse, Localized Features , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[42]  Josef Kittler,et al.  Discriminative Learning and Recognition of Image Set Classes Using Canonical Correlations , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.