Improved Classification with Semi-supervised Deep Belief Network

Abstract Classification problem is important for big data processing, and deep learning method named deep belief network (DBN) is successfully applied into classification. But traditional DBN is an unsupervised learning method, which leads to a gap between extracted features and concrete tasks. In this paper, a semi-supervised DBN (SSDBN) based on semi-supervised restricted Boltzmann machine (SSRBM) is proposed to shorten the gap and improve the accuracy of classification. Firstly, through introducing relevance constraint, supervised information is equivalently integrated into the learning process of restricted Boltzmann machine. Secondly, SSDBN-based model is constructed to improve the accuracy of classification problem. Finally, the proposed SSDBN is validated with hand-written digits classification standard dataset MNIST, and experimental results show that SSDBN outperforms traditional DBN and other models with respect to classification.

[1]  Geoffrey E. Hinton,et al.  Discovering Binary Codes for Documents by Learning Deep Generative Models , 2011, Top. Cogn. Sci..

[2]  Xiao Liu,et al.  Stacked deep polynomial network based representation learning for tumor classification with small ultrasound image dataset , 2016, Neurocomputing.

[3]  Cheng-Jian Lin,et al.  An entropy-based quantum neuro-fuzzy inference system for classification applications , 2007, Neurocomputing.

[4]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[5]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[6]  Nicolas Le Roux,et al.  Representational Power of Restricted Boltzmann Machines and Deep Belief Networks , 2008, Neural Computation.

[7]  Geoffrey E. Hinton.  A Practical Guide to Training Restricted Boltzmann Machines , 2012, Neural Networks: Tricks of the Trade.

[8]  Yan Liu,et al.  Discriminative deep belief networks for visual data classification , 2011, Pattern Recognit..

[9]  Xiao Liu,et al.  DeepChart: Combining deep convolutional networks and deep belief networks in chart classification , 2016, Signal Process..

[10]  Frank L. Lewis,et al.  Identification of nonlinear dynamical systems using multilayered neural networks , 1996, Autom..

[11]  Manesh Kokare,et al.  Human age classification using facial skin aging features and artificial neural network , 2016, Cognitive Systems Research.

[12]  Antonio J. Plaza,et al.  Subspace-Based Support Vector Machines for Hyperspectral Image Classification , 2015, IEEE Geoscience and Remote Sensing Letters.

[13]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[14]  Yuan Yan Tang,et al.  Hyperspectral Image Classification Based on Regularized Sparse Representation , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[15]  Yasser Shekofteh,et al.  MLP-based isolated phoneme classification using likelihood features extracted from reconstructed phase space , 2015, Eng. Appl. Artif. Intell..

[16]  Ayman M. Eldeib,et al.  Breast cancer classification using deep belief networks , 2016, Expert Syst. Appl..

[17]  Trac D. Tran,et al.  Hyperspectral Image Classification via Kernel Sparse Representation , 2013, IEEE Trans. Geosci. Remote. Sens..

[18]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[19]  Liangpei Zhang,et al.  An Adaptive Memetic Fuzzy Clustering Algorithm With Spatial Information for Remote Sensing Imagery , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.