Adaptive and Weighted Collaborative Representations for image classification

Abstract Recently, Zhang et al. (2011) proposed a classifier based on Collaborative Representations (CR) with Regularized Least Squares (CRC-RLS) for image face recognition. CRC-RLS can replace Sparse Representation (SR) based Classification (SRC) as a simple and fast alternative. With SR resulting from an l 1 -Regularized Least Squares decomposition, CR starts from an l 2 -Regularized Least Squares formulation. Moreover, it has an algebraic solution. We extend CRC-RLS to the case where the samples or features are weighted. Particularly, we consider weights based on the classification confidence for samples and the variance of feature channels. The Weighted Collaborative Representation Classifier (WCRC) improves the classification performance over that of the original formulation, while keeping the simplicity and the speed of the original CRC-RLS formulation. Moreover we investigate into query-adaptive WCRC formulations and kernelized extensions that show further performance improvements but come at the expense of increased computation time.

[1]  H. Zou.  The Adaptive Lasso and Its Oracle Properties , 2006 .

[2]  Jiawei Han,et al.  Efficient Kernel Discriminant Analysis via Spectral Regression , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[3]  Alexander Gammerman,et al.  Ridge Regression Learning Algorithm in Dual Variables , 1998, ICML.

[4]  Roderick J. A. Little,et al.  Statistical Analysis with Missing Data , 1988 .

[5]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[6]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[7]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[8]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[9]  David M. Blei,et al.  Exploiting Covariate Similarity in Sparse Regression via the Pairwise Elastic Net , 2010, AISTATS.

[10]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[11]  Qing-Song Xu,et al.  Grouping Variable Selection by Weight Fused Elastic Net for Multi-Collinear Data , 2012, Commun. Stat. Simul. Comput..

[12]  Johannes Stallkamp,et al.  The German Traffic Sign Recognition Benchmark: A multi-class classification competition , 2011, The 2011 International Joint Conference on Neural Networks.

[13]  R. Fisher.  THE STATISTICAL UTILIZATION OF MULTIPLE MEASUREMENTS , 1938 .

[14]  Luc Van Gool,et al.  Weighted collaborative representation and classification of images , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[15]  Hao Helen Zhang,et al.  ON THE ADAPTIVE ELASTIC-NET WITH A DIVERGING NUMBER OF PARAMETERS. , 2009, Annals of statistics.

[16]  Tao Xiong,et al.  A combined SVM and LDA approach for classification , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[17]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[18]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..

[19]  Zhang Yi,et al.  Collaborative neighbor representation based classification using l2-minimization approach , 2013, Pattern Recognit. Lett..

[20]  Francis R. Bach,et al.  Trace Lasso: a trace norm regularization for correlated designs , 2011, NIPS.

[21]  R. Penrose.  A Generalized inverse for matrices , 1955 .

[22]  Ting Wang,et al.  Kernel Sparse Representation-Based Classifier , 2012, IEEE Transactions on Signal Processing.

[23]  Russell V. Lenth,et al.  Statistical Analysis With Missing Data (2nd ed.) (Book) , 2004 .

[24]  Luc Van Gool,et al.  Sparse Representation Based Projections , 2011, BMVC.

[25]  Stephen P. Boyd,et al.  An Interior-Point Method for Large-Scale $\ell_1$-Regularized Least Squares , 2007, IEEE Journal of Selected Topics in Signal Processing.

[26]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[27]  Luc Van Gool,et al.  Iterative Nearest Neighbors for classification and dimensionality reduction , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Lei Zhang,et al.  Sparse representation or collaborative representation: Which helps face recognition? , 2011, 2011 International Conference on Computer Vision.

[29]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[31]  David Zhang,et al.  Relaxed collaborative representation for pattern classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[33]  A. Martínez,et al.  The AR face databasae , 1998 .

[34]  Z. John Daye,et al.  Shrinkage and model selection with correlated variables via weighted fusion , 2009, Comput. Stat. Data Anal..