Sparse Representation Using Deep Learning to Classify Multi-Class Complex Data

Extracting best feature set to reinforce discrimination is always a challenge in machine learning. In this paper, a method named General Locally Linear Combination (GLLC) is proposed to extract automatic features using a deep autoencoder and also to reconstruct a sample based on the other samples sparsely in a low-dimensional space. Extracting features along with the discrimination ability of the sparse models have created a robust classifier that shows simultaneous reduction in samples and features. To enhance the capability of this scheme, some feature sets from several layers of an autoencoder are combined and an extension of GLLC has been proposed that called here as Multi-modal General Locally Linear Combination. Although the main application of the proposed methods is in visual classification and face recognition, they have been used in other applications. Extensive experiments are conducted to demonstrate that the proposed algorithms gain high accuracy on various datasets and outperform the rival methods.

[1]  David G. Stork,et al.  Pattern Classification , 1973 .

[2]  Julien Mairal,et al.  Structured sparsity through convex optimization , 2011, ArXiv.

[3]  Avinash C. Kak,et al.  PCA versus LDA , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Yanjun Qi,et al.  Unsupervised Feature Learning by Deep Sparse Coding , 2013, SDM.

[5]  Ethem Alpaydin,et al.  Introduction to machine learning , 2004, Adaptive computation and machine learning.

[6]  Nishio Takayuki,et al.  Deep Learning Tutorial , 2018 .

[7]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[8]  Stephen J. Wright.  Coordinate descent algorithms , 2015, Mathematical Programming.

[9]  Yuxiao Hu,et al.  Learning a Spatially Smooth Subspace for Face Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Tieniu Tan,et al.  Feature Coding in Image Classification: A Comprehensive Study , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[12]  Cordelia Schmid,et al.  Semi-Local Affine Parts for Object Recognition , 2004, BMVC.

[13]  Cheng-Yuan Liou,et al.  Autoencoder for words , 2014, Neurocomputing.

[14]  Y. Nesterov.  Gradient methods for minimizing composite objective function , 2007 .

[15]  Ben Taskar,et al.  Joint covariate selection and joint subspace selection for multiple classification problems , 2010, Stat. Comput..

[16]  Shamik Sural,et al.  Segmentation and histogram generation using the HSV color space for image retrieval , 2002, Proceedings. International Conference on Image Processing.

[17]  Stan Z. Li,et al.  Face recognition using the nearest feature line method , 1999, IEEE Trans. Neural Networks.

[18]  James Demmel,et al.  Applied Numerical Linear Algebra , 1997 .

[19]  Emmanuel J. Candès,et al.  NESTA: A Fast and Accurate First-Order Method for Sparse Recovery , 2009, SIAM J. Imaging Sci..

[20]  Yoshua Bengio,et al.  On the Expressive Power of Deep Architectures , 2011, ALT.

[21]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[22]  Allen Y. Yang,et al.  On the Lagrangian biduality of sparsity minimization problems , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[23]  Hong Liu,et al.  An improved label fusion approach with sparse patch‐based representation for MRI brain image segmentation , 2017, Int. J. Imaging Syst. Technol..

[24]  Vladik Kreinovich,et al.  Why l1 Is a Good Approximation to l0: A Geometric Explanation , 2013 .

[25]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[27]  P. Zhao,et al.  The composite absolute penalties family for grouped and hierarchical variable selection , 2009, 0909.0411.

[28]  David J. Kriegman,et al.  Clustering appearances of objects under varying illumination conditions , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[29]  Ran He,et al.  A Regularized Correntropy Framework for Robust Pattern Recognition , 2011, Neural Computation.

[30]  Seiichi Ozawa,et al.  A Multitask Learning Model for Online Pattern Recognition , 2009, IEEE Transactions on Neural Networks.

[31]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[32]  Qi Zhu,et al.  CCEDA: building bridge between subspace projection learning and sparse representation-based classification , 2014 .

[33]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[34]  Massimiliano Pontil,et al.  Convex multi-task feature learning , 2008, Machine Learning.

[35]  Margit Antal,et al.  Keystroke Dynamics on Android Platform , 2015 .

[36]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[37]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[38]  Rich Caruana,et al.  Multitask Learning , 1998, Encyclopedia of Machine Learning and Data Mining.

[39]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  D. Donoho,et al.  Atomic Decomposition by Basis Pursuit , 2001 .

[41]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[42]  Rajesh P. N. Rao,et al.  Probabilistic Models of the Brain: Perception and Neural Function , 2002 .