Discriminative ICA model with reconstruction constraint for image classification

Independent Component Analysis (ICA) is an effective unsupervised tool to learn statistically independent representations. However, ICA is not only sensitive to whitening but also difficult to learn an over-complete basis set. Consequently, ICA with soft Reconstruction cost(RICA) was presented to learn sparse representations with over-complete basis even on unwhitened data. Nevertheless, this model may not be an optimal discriminative model for classification tasks, because it failed to consider the association between the training sample and its class. In this paper, we propose a supervised Discriminative ICA model with Reconstruction constraint for image classification, named DRICA. DRICA brings in class information to learn the over-complete basis by incorporating inhomogeneous representation cost constraint into the RICA framework. This constraint leads to partition the set of basis vectors into several subsets corresponding to the sample classes, where each subset could sparsely model data samples from the same class but not others. Therefore, the proposed ICA model can learn an over-complete basis and an optimal multi-class classifier jointly. Some experiments carried out on several standard image databases validate the effectiveness of DRICA for image classification.

[1]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[2]  Larry S. Davis,et al.  Learning a discriminative dictionary for sparse coding via label consistent K-SVD , 2011, CVPR 2011.

[3]  E. Oja,et al.  Independent Component Analysis , 2001 .

[4]  Yihong Gong,et al.  Linear spatial pyramid matching using sparse coding for image classification , 2009, CVPR.

[5]  Tong Zhang,et al.  Improved Local Coordinate Coding using Local Tangents , 2010, ICML.

[6]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[7]  Baoxin Li,et al.  Discriminative K-SVD for dictionary learning in face recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  Quoc V. Le,et al.  ICA with Reconstruction Cost for Efficient Overcomplete Feature Learning , 2011, NIPS.

[9]  Quoc V. Le,et al.  Tiled convolutional neural networks , 2010, NIPS.

[10]  A. Krizhevsky.  Convolutional Deep Belief Networks on CIFAR-10 , 2010 .

[11]  Erkki Oja,et al.  Independent Component Analysis , 2001 .

[12]  Honglak Lee,et al.  An Analysis of Single-Layer Networks in Unsupervised Feature Learning , 2011, AISTATS.

[13]  Thomas Hofmann,et al.  Greedy Layer-Wise Training of Deep Networks , 2007 .

[14]  Aapo Hyvärinen,et al.  Natural Image Statistics - A Probabilistic Approach to Early Computational Vision , 2009, Computational Imaging and Vision.