Semi-supervised convolutional extreme learning machine

We propose a scheme for training a neural network as an image classifier. The approach includes a very rapid unsupervised feature learning algorithm and a supervised technique. We show that convolving and downsampling clustered descriptors of image patches with each input image can provide more discriminative features compared to both pre-trained descriptors and randomly generated convolutional filters. The implemented algorithm to discover clusters centroids (i.e. k-means clustering) for color images is not restricted to only RGB and we show that the algorithm is appropriate for Lab color representations. We use the centroids for obtaining convolutional features. We also present a high performance extreme learning machine (ELM), which is a method characterized by low implementation complexity, and run-time, to classify the learned features. We show that the combination of the unsupervised feature learning with the ELM outperforms previous related models that use different feature representations fed into an ELM, on the CIFAR-10 and Google Street View House Number (SVHN) datasets.

[1]  Pedro M. Domingos,et al.  Discriminative Learning of Sum-Product Networks , 2012, NIPS.

[2]  Shai Shalev-Shwartz,et al.  K-means recovers ICA filters when independent components are sparse , 2014, ICML.

[3]  Jun Miao,et al.  > Replace This Line with Your Paper Identification Number (double-click Here to Edit) < , 2022 .

[4]  Zhuowen Tu,et al.  Generalizing Pooling Functions in Convolutional Neural Networks: Mixed, Gated, and Tree , 2015, AISTATS.

[5]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[6]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[7]  Andrew Zisserman,et al.  A Statistical Approach to Material Classification Using Image Patch Exemplars , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Mark D. McDonnell,et al.  Enhanced image classification with a fast-learning shallow convolutional neural network , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[9]  Jun Miao,et al.  Constrained Extreme Learning Machine: A novel highly discriminative random feedforward neural network , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[10]  Andrew Y. Ng,et al.  Reading Digits in Natural Images with Unsupervised Feature Learning , 2011 .

[11]  Chi-Man Vong,et al.  Local Receptive Fields Based Extreme Learning Machine , 2015, IEEE Computational Intelligence Magazine.

[12]  Weihua Liu,et al.  The effect of whitening transformation on pooling operations in convolutional autoencoders , 2015, EURASIP J. Adv. Signal Process..

[13]  R. Hunter.  Photoelectric Color Difference Meter , 1958 .

[14]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[15]  Robert P. W. Duin,et al.  Feedforward neural networks with random weights , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol.II. Conference B: Pattern Recognition Methodology and Systems.

[16]  Mark D. McDonnell,et al.  On the importance of pair-wise feature correlations for image classification , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[17]  Benjamin Graham,et al.  Fractional Max-Pooling , 2014, ArXiv.

[18]  Honglak Lee,et al.  An Analysis of Single-Layer Networks in Unsupervised Feature Learning , 2011, AISTATS.

[19]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[20]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[21]  Yann LeCun,et al.  What is the best multi-stage architecture for object recognition? , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[22]  Lior Wolf,et al.  Patch-Based Texture Edges and Segmentation , 2006, ECCV.

[23]  Bin Li,et al.  Gaussian message passing-based cooperative localization on factor graph in wireless networks , 2015, Signal Process..

[24]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[25]  Donald C. Wunsch,et al.  Unsupervised Feature Learning Classification With Radial Basis Function Extreme Learning Machine Using Graphic Processors , 2017, IEEE Transactions on Cybernetics.

[26]  Yihong Gong,et al.  Linear spatial pyramid matching using sparse coding for image classification , 2009, CVPR.

[27]  Sepp Hochreiter,et al.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.

[28]  Guang-Bin Huang,et al.  An Insight into Extreme Learning Machines: Random Neurons, Random Features and Kernels , 2014, Cognitive Computation.

[29]  Andrew Y. Ng,et al.  Learning Feature Representations with K-Means , 2012, Neural Networks: Tricks of the Trade.

[30]  Donald C. Wunsch,et al.  Unsupervised feature learning classification using an extreme learning machine , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[31]  Mark D. McDonnell,et al.  Fast, Simple and Accurate Handwritten Digit Classification by Training Shallow Neural Network Classifiers with the ‘Extreme Learning Machine’ Algorithm , 2015, PloS one.

[32]  Yann LeCun,et al.  Convolutional neural networks applied to house numbers digit classification , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[33]  Yihong Gong,et al.  Locality-constrained Linear Coding for image classification , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[34]  Sergei Vassilvitskii,et al.  k-means++: the advantages of careful seeding , 2007, SODA '07.

[35]  Guang-Bin Huang,et al.  Extreme Learning Machine for Multilayer Perceptron , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[36]  Thomas Brox,et al.  Discriminative Unsupervised Feature Learning with Convolutional Neural Networks , 2014, NIPS.