Learning Discrimination Specific, Self-Collaborative and Nonlinear Model

This paper presents a novel nonlinear transform model for learning of collaboration structured, discriminative and sparse representations. The idea is to model a collaboration corrective functionality between multiple nonlinear transforms in order to reduce the uncertainty in the estimate. The focus is on the joint estimation of data-adaptive nonlinear transforms (NTs) that take into account a collaboration component w.r.t. a discrimination target. The joint model includes minimum information loss, collaboration corrective and discriminative priors. The model parameters are learned by minimizing an approximation to the empirical negative log likelihood of the model, where we propose an efficient solution by an iterative, coordinate descent algorithm. Numerical experiments validate the potential of the learning principle. The preliminary results show advantages in comparison to the stateof-the-art methods, w.r.t. the learning time, the discriminative quality and the recognition accuracy.

[1]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[3]  Benjamin Graham,et al.  Fractional Max-Pooling , 2014, ArXiv.

[4]  Maheshkumar H. Kolekar,et al.  Classification of fashion article images using convolutional neural networks , 2017, 2017 Fourth International Conference on Image Information Processing (ICIIP).

[5]  Vishal Monga,et al.  Fast Low-Rank Shared Dictionary Learning for Image Classification , 2016, IEEE Transactions on Image Processing.

[6]  Pierre Baldi,et al.  Autoencoders, Unsupervised Learning, and Deep Architectures , 2011, ICML Unsupervised and Transfer Learning.

[7]  Patrick Seemann,et al.  Matrix Factorization Techniques for Recommender Systems , 2014 .

[8]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[9]  Vishal Monga,et al.  DFDL: Discriminative feature-oriented dictionary learning for histopathological image classification , 2015, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).

[10]  Y. LeCun,et al.  Learning methods for generic object recognition with invariance to pose and lighting , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[11]  Roland Vollgraf,et al.  Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms , 2017, ArXiv.

[12]  Guillermo Sapiro,et al.  Classification and clustering via dictionary learning with structured incoherence and shared features , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[14]  David Zhang,et al.  Fisher Discrimination Dictionary Learning for sparse representation , 2011, 2011 International Conference on Computer Vision.

[15]  Yoram Bresler,et al.  Closed-form solutions within sparsifying transform learning , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[16]  Dimche Kostadinov,et al.  Learning non-structured , overcomplete and sparsifying transform , 2017 .

[17]  Andreas Krause,et al.  Discriminative Clustering by Regularized Information Maximization , 2010, NIPS.

[18]  Michael Elad,et al.  Analysis K-SVD: A Dictionary-Learning Algorithm for the Analysis Sparse Model , 2013, IEEE Transactions on Signal Processing.

[19]  Yoram Bresler,et al.  Learning sparsifying transforms for image processing , 2012, 2012 19th IEEE International Conference on Image Processing.

[20]  P. Bühlmann,et al.  Statistical Applications in Genetics and Molecular Biology Low-Order Conditional Independence Graphs for Inferring Genetic Networks , 2011 .

[21]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[22]  Silvio Savarese,et al.  Hierarchical classification of images by sparse approximation , 2013, Image Vis. Comput..

[23]  Michael Elad,et al.  Analysis versus synthesis in signal priors , 2006, 2006 14th European Signal Processing Conference.

[24]  Brendan J. Frey,et al.  Winner-Take-All Autoencoders , 2014, NIPS.

[25]  Dale Schuurmans,et al.  Maximum Margin Clustering , 2004, NIPS.

[26]  Pascal Frossard,et al.  Structured sparse coding for image denoising or pattern detection , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[27]  Zaïd Harchaoui,et al.  DIFFRAC: a discriminative and flexible framework for clustering , 2007, NIPS.

[28]  Michael I. Jordan,et al.  Learning Spectral Clustering, With Application To Speech Separation , 2006, J. Mach. Learn. Res..

[29]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .

[30]  Andrew Y. Ng,et al.  Reading Digits in Natural Images with Unsupervised Feature Learning , 2011 .

[31]  A. Martínez,et al.  The AR face databasae , 1998 .

[32]  Esa Alhoniemi,et al.  Clustering of the self-organizing map , 2000, IEEE Trans. Neural Networks Learn. Syst..

[33]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[34]  Kaushik Roy,et al.  Unsupervised regenerative learning of hierarchical features in Spiking Deep Networks for object recognition , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[35]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[36]  Michael Elad,et al.  Dictionary Learning for Analysis-Synthesis Thresholding , 2014, IEEE Transactions on Signal Processing.

[37]  Chris Eliasmith,et al.  Spiking Deep Networks with LIF Neurons , 2015, ArXiv.

[38]  Chih-Jen Lin,et al.  A Comparison of Methods for Multi-class Support Vector Machines , 2015 .

[39]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[40]  Klaus Diepold,et al.  Analysis Operator Learning and its Application to Image Reconstruction , 2012, IEEE Transactions on Image Processing.

[41]  Zhuowen Tu,et al.  Generalizing Pooling Functions in Convolutional Neural Networks: Mixed, Gated, and Tree , 2015, AISTATS.

[42]  Michael Elad,et al.  Dictionaries for Sparse Representation Modeling , 2010, Proceedings of the IEEE.

[43]  Volkan Cevher,et al.  Model-Based Compressive Sensing , 2008, IEEE Transactions on Information Theory.