Convolutional Neural Network Features Comparison Between Back-Propagation and Extreme Learning Machine

Recently deep learning based architectures have been widely deployed in many problems of artificial intelligence. Among deep learning models, Convolutional Neural Networks (CNN) have been reported in numerous successful applications such as object recognition, and natural language processing. The convolutional neural networks are trained by back-propagating the classification error using the Back-Propagation (BP) algorithm, which requires a large amount of data and slows the training process. To overcome these difficulties, a new fast and accurate approach based on Extreme Learning Machine (ELM) to train any convolutional neural network has been proposed. The developed framework (ELM-CNN) is based on the concept of auto-encoding to learn the convolutional filters with biases, by reconstructing the normalized input and the intercept term. In this paper, systematic comparison with traditional back-propagation based training method (BP-CNN) has been made with respect to two aspects qualitative and quantitative. The experimental results on the popular MNIST dataset show that the ELM-CNN algorithm achieves competitive results in terms of generalization performance and up to 16 times faster than the back-propagation based training of CNN.