Image classification with stacked restricted boltzmann machines and evolutionary function array classification voter

Deep belief networks (DBNs) are graphical models which learn to extract the deep hierarchical representation of the input data. Restricted boltzmann machines can be stacked and trained in a greedy manner to form deep belief networks. The combination of deep belief networks and the classifier is used for data classification. However, with the increase of the size of the training set, it requires a huge amount of training time to get better classification accuracy results. To address these issues, a novel fast classifier, called evolutionary function array classification voter (EFACV) is proposed in this research. EFACV is inspired by evolution hardware and it is combined with deep belief networks which is formed by a stack of restricted boltzmann machines for image classification. In our approach, the stacked restricted boltzmann machines are used to extract the feature vectors of the images and EFACV is used to classify these feature vectors. Experimental verifications are conducted on MNIST dataset. Experimental results indicate that compared with the combination of the stacked restricted boltzmann machines and softmax classifier and the combination of the stacked restricted boltzmann machines and support vector machine, our approach can get the higher classification accuracy in less time and have superior anti-overfitting ability.

[1]  Moncef Gabbouj,et al.  Collective network of evolutionary binary classifiers for content-based image retrieval , 2011, 2011 IEEE Workshop on Evolving and Adaptive Intelligent Systems (EAIS).

[2]  Chun-Chieh Yang,et al.  Application of decision tree technology for image classification using remote sensing data , 2003 .

[3]  Honglak Lee,et al.  Sparse deep belief net model for visual area V2 , 2007, NIPS.

[4]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[5]  Thomas Bäck,et al.  An Overview of Evolutionary Algorithms for Parameter Optimization , 1993, Evolutionary Computation.

[6]  V. Sowmya,et al.  Variational Mode Feature-Based Hyperspectral Image Classification , 2015 .

[7]  Tong Heng Lee,et al.  A distributed evolutionary classifier for knowledge discovery in data mining , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[8]  Robert Sabourin,et al.  Dynamic multi-objective evolution of classifier ensembles for video face recognition , 2013, Appl. Soft Comput..

[9]  Cunwei Lu,et al.  3-D face recognition method based on optimum 3-D image measurement technology , 2011, Artificial Life and Robotics.

[10]  Antonio J. Plaza,et al.  A Quantitative and Comparative Assessment of Unmixing-Based Feature Extraction Techniques for Hyperspectral Image Classification , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[11]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[12]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[13]  Geoffrey E. Hinton,et al.  Restricted Boltzmann machines for collaborative filtering , 2007, ICML '07.

[14]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[15]  Geoffrey E. Hinton,et al.  Factored 3-Way Restricted Boltzmann Machines For Modeling Natural Images , 2010, AISTATS.

[16]  Helen M. Meng,et al.  Multi-distribution deep belief network for speech synthesis , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[17]  Isamu Kajitani,et al.  Evolvable hardware with genetic learning , 1996, 1996 IEEE International Symposium on Circuits and Systems. Circuits and Systems Connecting the World. ISCAS 96.

[18]  Peijun Du,et al.  Multiple Classifier Ensembles with Band Clustering for Hyperspectral Image Classification , 2014 .

[19]  Antonio Torralba,et al.  Small codes and large image databases for recognition , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Geoffrey E. Hinton,et al.  Replicated Softmax: an Undirected Topic Model , 2009, NIPS.

[21]  Zhenghao Chen,et al.  On Random Weights and Unsupervised Feature Learning , 2011, ICML.

[22]  Ming Yang,et al.  Large-scale image classification: Fast feature extraction and SVM training , 2011, CVPR 2011.

[23]  Jin Wang,et al.  Design and implementation of a virtual reconfigurable architecture for different applications of intrinsic evolvable hardware , 2008, IET Comput. Digit. Tech..

[24]  Marek Kurzynski,et al.  On the multistage Bayes classifier , 1988, Pattern Recognit..

[25]  Chiang Wei,et al.  A Feature-Space Indicator Kriging Approach for Remote Sensing Image Classification , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Terence C. Fogarty,et al.  Co-evolutionary classifier systems for multi-agent simulation , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[27]  Jiasong Zhu,et al.  Discriminative Gabor Feature Selection for Hyperspectral Image Classification , 2013, IEEE Geoscience and Remote Sensing Letters.