Unsupervised Deep Transform Learning

We introduce deep transform learning - a new tool for deep learning. Deeper representation is learnt by stacking one transform after another. The model is akin to a feedforward neural network. The first layer learns the transform and features from the input training samples. Subsequent layers use the features (after activation) from the previous layers as training input. However, this explanation is only given for intuitive understanding; the ensuing problem is not solved in a greedy fashion. All the layers are solved jointly. Experiments have been carried out with other state-of-the-art. Results on classification and clustering show that our proposed technique is better than all the said techniques, at least on the benchmark datasets compared on.

[1]  Xue-Cheng Tai,et al.  Augmented Lagrangian Method, Dual Methods, and Split Bregman Iteration for ROF, Vectorial TV, and High Order Models , 2010, SIAM J. Imaging Sci..

[2]  Yoram Bresler,et al.  Online Sparsifying Transform Learning— Part I: Algorithms , 2015, IEEE Journal of Selected Topics in Signal Processing.

[3]  Jyoti Maggu,et al.  Robust transform learning , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  José M. Bioucas-Dias,et al.  Multiplicative Noise Removal Using Variable Splitting and Constrained Optimization , 2009, IEEE Transactions on Image Processing.

[5]  Yoram Bresler,et al.  Online Sparsifying Transform Learning—Part II: Convergence Analysis , 2015, IEEE Journal of Selected Topics in Signal Processing.

[6]  Rama Chellappa,et al.  Multiple Kernel Learning for Sparse Representation-Based Classification , 2014, IEEE Transactions on Image Processing.

[7]  José M. F. Moura,et al.  Linear Convergence Rate of a Class of Distributed Augmented Lagrangian Algorithms , 2013, IEEE Transactions on Automatic Control.

[8]  Yoram Bresler,et al.  Learning Sparsifying Transforms , 2013, IEEE Transactions on Signal Processing.

[9]  Angshul Majumdar,et al.  Majorization Minimization Technique for Optimally Solving Deep Dictionary Learning , 2017, Neural Processing Letters.

[10]  Rama Chellappa,et al.  Analysis sparse coding models for image-based classification , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[11]  Yoram Bresler,et al.  Efficient Blind Compressed Sensing Using Sparsifying Transforms with Convergence Guarantees and Application to Magnetic Resonance Imaging , 2015, SIAM J. Imaging Sci..

[12]  Richa Singh,et al.  Detecting Silicone Mask-Based Presentation Attack via Deep Dictionary Learning , 2017, IEEE Transactions on Information Forensics and Security.

[13]  Rama Chellappa,et al.  Kernel dictionary learning , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[14]  Karthikeyan Natesan Ramamurthy,et al.  Learning Stable Multilevel Dictionaries for Sparse Representations , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[15]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[16]  Dao-Qing Dai,et al.  Learning Kernel Extended Dictionary for Face Recognition , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[17]  Mayank Vatsa,et al.  Deep Dictionary Learning , 2016, IEEE Access.

[18]  Jian Yang,et al.  A Locality-Constrained and Label Embedding Dictionary Learning Algorithm for Image Classification , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[19]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[20]  F. Cao,et al.  Image Super-Resolution via Adaptive $\ell _{p} (0, 2016, IEEE Transactions on Neural Networks and Learning Systems.

[21]  Angshul Majumdar,et al.  Discriminative Robust Deep Dictionary Learning for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[22]  Michael K. Ng,et al.  Dictionary Learning-Based Subspace Structure Identification in Spectral Clustering , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[23]  Angshul Majumdar,et al.  Deep Sparse Coding for Non–Intrusive Load Monitoring , 2018, IEEE Transactions on Smart Grid.

[24]  Jyoti Maggu,et al.  Greedy deep transform learning , 2017, 2017 IEEE International Conference on Image Processing (ICIP).