Neural-Response-Based Extreme Learning Machine for Image Classification

This paper proposes a novel and simple multilayer feature learning method for image classification by employing the extreme learning machine (ELM). The proposed algorithm is composed of two stages: the multilayer ELM (ML-ELM) feature mapping stage and the ELM learning stage. The ML-ELM feature mapping stage is recursively built by alternating between feature map construction and maximum pooling operation. In particular, the input weights for constructing feature maps are randomly generated and hence need not be trained or tuned, which makes the algorithm highly efficient. Moreover, the maximum pooling operation enables the algorithm to be invariant to certain transformations. During the ELM learning stage, elastic-net regularization is proposed to learn the output weight. Elastic-net regularization helps to learn more compact and meaningful output weight. In addition, we preprocess the input data with the dense scale-invariant feature transform operation to improve both the robustness and invariance of the algorithm. To evaluate the effectiveness of the proposed method, several experiments are conducted on three challenging databases. Compared with the conventional deep learning methods and other related ones, the proposed method achieves the best classification results with high computational efficiency.

[1]  Geoffrey E. Hinton,et al.  Deep Boltzmann Machines , 2009, AISTATS.

[2]  Yihong Gong,et al.  Linear spatial pyramid matching using sparse coding for image classification , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Robert P. W. Duin,et al.  Feedforward neural networks with random weights , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol.II. Conference B: Pattern Recognition Methodology and Systems.

[4]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[5]  Hongkai Zhao,et al.  Variational Hamiltonian Monte Carlo via Score Matching. , 2016, Bayesian analysis.

[6]  Guang-Bin Huang,et al.  Extreme Learning Machine for Multilayer Perceptron , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[7]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[8]  Stefano Fusi,et al.  The Sparseness of Mixed Selectivity Neurons Controls the Generalization–Discrimination Trade-Off , 2013, The Journal of Neuroscience.

[9]  Bo Liu,et al.  Image classification based on effective extreme learning machine , 2013, Neurocomputing.

[10]  Guang-Bin Huang,et al.  Convex incremental extreme learning machine , 2007, Neurocomputing.

[11]  Donald W. Marquaridt.  Generalized Inverses, Ridge Regression, Biased Linear Estimation, and Nonlinear Estimation , 1970 .

[12]  Tara N. Sainath,et al.  FUNDAMENTAL TECHNOLOGIES IN MODERN SPEECH RECOGNITION Digital Object Identifier 10.1109/MSP.2012.2205597 , 2012 .

[13]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[14]  Ah Chung Tsoi,et al.  Face recognition: a convolutional neural-network approach , 1997, IEEE Trans. Neural Networks.

[15]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[16]  L. C. Kasun,et al.  Representational Learning with Extreme Learning Machine for Big Data Liyanaarachchi , 2022 .

[17]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[18]  Na Chen,et al.  Error Analysis for Matrix Elastic-Net Regularization Algorithms , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[20]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[21]  Guang-Bin Huang,et al.  An Insight into Extreme Learning Machines: Random Neurons, Random Features and Kernels , 2014, Cognitive Computation.

[22]  Halbert White,et al.  Approximate Nonlinear Forecasting Methods , 2006 .

[23]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[24]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[25]  Rasmus Berg Palm,et al.  Prediction as a candidate for learning deep hierarchical models of data , 2012 .

[26]  Yihong Gong,et al.  Locality-constrained Linear Coding for image classification , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[27]  Quoc V. Le,et al.  Tiled convolutional neural networks , 2010, NIPS.

[28]  H. White,et al.  An additional hidden unit test for neglected nonlinearity in multilayer feedforward networks , 1989, International 1989 Joint Conference on Neural Networks.

[29]  Yicong Zhou,et al.  Extreme Learning Machine With Composite Kernels for Hyperspectral Image Classification , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[30]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[31]  Hongming Zhou,et al.  Extreme Learning Machine based fast object recognition , 2012, 2012 15th International Conference on Information Fusion.

[32]  Chi-Man Vong,et al.  Local Receptive Fields Based Extreme Learning Machine , 2015, IEEE Computational Intelligence Magazine.

[33]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[34]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[35]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[36]  C. L. Philip Chen,et al.  Hierarchical Feature Extraction With Local Neural Response for Image Recognition , 2013, IEEE Transactions on Cybernetics.

[37]  Yantao Wei,et al.  Similarity learning for object recognition based on derived kernel , 2012, Neurocomputing.

[38]  Cheng Wu,et al.  Semi-Supervised and Unsupervised Extreme Learning Machines , 2014, IEEE Transactions on Cybernetics.

[39]  Lei Chen,et al.  Enhanced random search based incremental extreme learning machine , 2008, Neurocomputing.

[40]  Thomas Serre,et al.  Robust Object Recognition with Cortex-Like Mechanisms , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Lorenzo Rosasco,et al.  Publisher Accessed Terms of Use Detailed Terms Mathematics of the Neural Response , 2022 .

[42]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[43]  Guang-Bin Huang,et al.  Trends in extreme learning machines: A review , 2015, Neural Networks.

[44]  Guang-Bin Huang,et al.  Extreme learning machine: a new learning scheme of feedforward neural networks , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[45]  Babak Shahbaba,et al.  Hamiltonian Monte Carlo acceleration using surrogate functions with random bases , 2015, Statistics and Computing.

[46]  Q. M. Jonathan Wu,et al.  Incremental Learning in Human Action Recognition Based on Snippets , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[47]  Yuan Yan Tang,et al.  Sparse-based neural response for image classification , 2014, Neurocomputing.