Graph regularized ICA for over-complete feature learning

Independent Component Analysis with a soft Reconstruction cost (RICA) has been recently presented to learn highly over-complete sparse features even on unwhitened data. However, RICA failed to consider the geometrical structure of the data space, which has been shown essential for classification problems. To address this problem, we propose a graph regularized ICA model with Reconstruction constraint for image classification, called gRICA. In particular, we construct an affinity graph to encode the geometrical information, and thereby learn a graph regularized over-complete basis which makes sparse representations respect the graph structure. Experiments conducted on several datasets show the effectiveness of gRICA for classification.

[1]  E. Oja,et al.  Independent Component Analysis , 2001 .

[2]  Fan Chung,et al.  Spectral Graph Theory , 1996 .

[3]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[4]  Larry S. Davis,et al.  Learning a discriminative dictionary for sparse coding via label consistent K-SVD , 2011, CVPR 2011.

[5]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[6]  Yihong Gong,et al.  Linear spatial pyramid matching using sparse coding for image classification , 2009, CVPR.

[7]  Tong Zhang,et al.  Improved Local Coordinate Coding using Local Tangents , 2010, ICML.

[8]  Yoshua Bengio,et al.  Greedy Layer-Wise Training of Deep Networks , 2006, NIPS.

[9]  Honglak Lee,et al.  An Analysis of Single-Layer Networks in Unsupervised Feature Learning , 2011, AISTATS.

[10]  Chun Chen,et al.  Graph Regularized Sparse Coding for Image Representation , 2011, IEEE Transactions on Image Processing.

[11]  Aapo Hyvärinen,et al.  Natural Image Statistics - A Probabilistic Approach to Early Computational Vision , 2009, Computational Imaging and Vision.

[12]  Quoc V. Le,et al.  ICA with Reconstruction Cost for Efficient Overcomplete Feature Learning , 2011, NIPS.

[13]  Thomas S. Huang,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation. , 2011, IEEE transactions on pattern analysis and machine intelligence.

[14]  Quoc V. Le,et al.  Tiled convolutional neural networks , 2010, NIPS.