Image Classification Using Deep Autoencoders

Deep learning refers to computational models comprising of several processing layers that permit display of data with a compound level of abstraction. Till date, several deep learning architectures have been developed, and notable results are attained. The best result often involves an unsupervised pretraining phase followed by supervised learning task. In this work, a particular implementation of deep autoencoders with SVM (Support Vector Machine) layer as a classification layer on the top of the encoding layer is explored. A comparison is made on MNIST dataset with softmax regression function layer and SVM layer as a classification layer with 2 layers and 3 layers SAE (Stack Autoencoders) respectively. Experimental results are evaluated using SAE. It is observed that SVM as classification layer obtains 99.8% accuracy with 0.2% error rate and outperforms softmax regression layer as a classification layer in autoencoders. Further, affect of varying number of neurons in the hidden layers of the autoencoders on the network performance with regard to classification accuracy is also studied.

[1]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[2]  Pieter Abbeel,et al.  InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets , 2016, NIPS.

[3]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[4]  © Tan,et al.  Data Mining Classification : Basic Concepts , Decision Trees , and Model Evaluation , 2004 .

[5]  Parviz Keshavarzi,et al.  A novel MLP network implementation in CMOL technology , 2014 .

[6]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[7]  Charles Blundell,et al.  Early Visual Concept Learning with Unsupervised Deep Learning , 2016, ArXiv.

[8]  S. Karsoliya,et al.  Approximating Number of Hidden layer neurons in Multiple Hidden Layer BPNN Architecture , 2012 .

[9]  Pierre Baldi,et al.  Autoencoders, Unsupervised Learning, and Deep Architectures , 2011, ICML Unsupervised and Transfer Learning.

[10]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[11]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[12]  Thomas Hofmann,et al.  Greedy Layer-Wise Training of Deep Networks , 2007 .

[13]  Razvan Pascanu,et al.  Advances in optimizing recurrent networks , 2012, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[14]  Joshua B. Tenenbaum,et al.  Understanding Visual Concepts with Continuation Learning , 2016, ArXiv.

[15]  Chris Varano,et al.  Disentangling Variational Autoencoders for Image Classification , 2017 .

[16]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Nishio Takayuki,et al.  Deep Learning Tutorial , 2018 .

[18]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .

[19]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[20]  Max Welling,et al.  Semi-supervised Learning with Deep Generative Models , 2014, NIPS.