Sampling Representative Examples for Dimensionality Reduction and Recognition - Bootstrap Bumping LDA

We present a novel method for dimensionality reduction and recognition based on Linear Discriminant Analysis (LDA), which specifically deals with the Small Sample Size (SSS) problem in Computer Vision applications. Unlike the traditional methods, which impose specific assumptions to address the SSS problem, our approach introduces a variant of bootstrap bumping technique, which is a general framework in statistics for model search and inference. An intermediate linear representation is first hypothesized from each bootstrap sample. Then LDA is performed in the reduced subspace. Lastly, the final model is selected among all hypotheses for the best classification. Experiments on synthetic and real datasets demonstrate the advantages of our Bootstrap Bumping LDA (BB-LDA) approach over the traditional LDA based methods.

[1]  James W. Davis,et al.  The representation and recognition of human movement using temporal templates , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[3]  C. R. Rao,et al.  The Utilization of Multiple Measurements in Problems of Biological Classification , 1948 .

[4]  Robert E. Schapire,et al.  The strength of weak learnability , 1990, Mach. Learn..

[5]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[7]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[8]  Takeo Kanade,et al.  Oriented Discriminant Analysis , 2004, BMVC.

[9]  M. Nixon,et al.  Human gait recognition in canonical space using temporal templates , 1999 .

[10]  W. V. McCarthy,et al.  Discriminant Analysis with Singular Covariance Matrices: Methods and Applications to Spectroscopic Data , 1995 .

[11]  G. Baudat,et al.  Generalized Discriminant Analysis Using a Kernel Approach , 2000, Neural Computation.

[12]  N. Campbell.  CANONICAL VARIATE ANALYSIS—A GENERAL MODEL FORMULATION , 1984 .

[13]  Takeo Kanade,et al.  Oriented Discriminant Analysis (ODA) , 2004 .

[14]  J. Friedman.  Regularized Discriminant Analysis , 1989 .

[15]  Robert P. W. Duin,et al.  Bagging, Boosting and the Random Subspace Method for Linear Classifiers , 2002, Pattern Analysis & Applications.

[16]  Paul A. Viola,et al.  Detecting Pedestrians Using Patterns of Motion and Appearance , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[17]  Chengjun Liu,et al.  Enhanced Fisher linear discriminant models for face recognition , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[18]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[19]  James W. Davis,et al.  The Representation and Recognition of Action Using Temporal Templates , 1997, CVPR 1997.

[20]  Josef Kittler,et al.  Locally linear discriminant analysis for multimodally distributed classes for face recognition with a single model image , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Ralph Gross,et al.  The CMU Motion of Body (MoBo) Database , 2001 .

[22]  David G. Stork,et al.  Pattern Classification , 1973 .

[23]  Hui Gao,et al.  Why direct LDA is not equivalent to LDA , 2006, Pattern Recognit..

[24]  Anil K. Jain,et al.  Resampling for Face Recognition , 2003, AVBPA.

[25]  Hua Yu,et al.  A direct LDA algorithm for high-dimensional data - with application to face recognition , 2001, Pattern Recognit..

[26]  Robert Tibshirani,et al.  An Introduction to the Bootstrap , 1994 .

[27]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[28]  Yoav Freund,et al.  Boosting a weak learning algorithm by majority , 1990, COLT '90.

[29]  Anuj Srivastava,et al.  Optimal linear representations of images for object recognition , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[30]  David Hinkley,et al.  Bootstrap Methods: Another Look at the Jackknife , 2008 .

[31]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[32]  Yuntao Cui,et al.  Learning-based hand sign recognition using SHOSLIF-M , 1995, Proceedings of IEEE International Conference on Computer Vision.

[33]  R. Tibshirani,et al.  Model Search by Bootstrap “Bumping” , 1999 .