Recognition of Handwritten Digit using Convolutional Neural Network in Python with Tensorflow and Comparison of Performance for Various Hidden Layers

In recent times, with the increase of Artificial Neural Network (ANN), deep learning has brought a dramatic twist in the field of machine learning by making it more artificially intelligent. Deep learning is remarkably used in vast ranges of fields because of its diverse range of applications such as surveillance, health, medicine, sports, robotics, drones, etc. In deep learning, Convolutional Neural Network (CNN) is at the center of spectacular advances that mixes Artificial Neural Network (ANN) and up to date deep learning strategies. It has been used broadly in pattern recognition, sentence classification, speech recognition, face recognition, text categorization, document analysis, scene, and handwritten digit recognition. The goal of this paper is to observe the variation of accuracies of CNN to classify handwritten digits using various numbers of hidden layers and epochs and to make the comparison between the accuracies. For this performance evaluation of CNN, we performed our experiment using Modified National Institute of Standards and Technology (MNIST) dataset. Further, the network is trained using stochastic gradient descent and the backpropagation algorithm. Keywords—Handwritten digit recognition, Convolutional Neural Network (CNN), Deep learning, MNIST dataset, Epochs, Hidden Layers, Stochastic Gradient Descent, Backpropagation

[1]  Simon Haykin,et al.  GradientBased Learning Applied to Document Recognition , 2001 .

[2]  Sargur N. Srihari,et al.  Fast k-nearest neighbor classification using cluster-based trees , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  D. Hubel,et al.  Aberrant visual projections in the Siamese cat , 1971, The Journal of physiology.

[4]  Robert Hecht-Nielsen,et al.  Theory of the backpropagation neural network , 1989, International 1989 Joint Conference on Neural Networks.

[5]  Qingshan Liu,et al.  Convolutional neural networks with large-margin softmax loss function for cognitive load recognition , 2017, 2017 36th Chinese Control Conference (CCC).

[6]  김창욱,et al.  A Convolutional Neural Network for Fault Classification and Diagnosis in Semiconductor Manufacturing Processes , 2016 .

[7]  Qi Tian,et al.  DisturbLabel: Regularizing CNN on the Loss Layer , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Yan Yin,et al.  Ncfm: Accurate handwritten digits recognition using Convolutional Neural Networks , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[9]  Kunihiko Fukushima,et al.  Neocognitron: A Self-Organizing Neural Network Model for a Mechanism of Visual Pattern Recognition , 1982 .

[10]  Changshui Zhang,et al.  Traffic Sign Recognition With Hinge Loss Trained Convolutional Neural Networks , 2014, IEEE Transactions on Intelligent Transportation Systems.

[11]  Takashi Ida,et al.  Deep Shrinkage Convolutional Neural Network for Adaptive Noise Reduction , 2018, IEEE Signal Processing Letters.

[12]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .

[13]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[14]  Md. Abu Bakr Siddique,et al.  Study and Observation of the Variations of Accuracies for Handwritten Digits Recognition with Various Hidden Layers and Epochs using Convolutional Neural Network , 2018, 2018 4th International Conference on Electrical Engineering and Information & Communication Technology (iCEEiCT).

[15]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[16]  Jürgen Schmidhuber,et al.  Multi-column deep neural networks for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Charu C. Aggarwal,et al.  Neural Networks and Deep Learning , 2018, Springer International Publishing.

[18]  Md. Abu Bakr Siddique,et al.  Study and Observation of the Variations of Accuracies for Handwritten Digits Recognition with Various Hidden Layers and Epochs using Neural Network Algorithm , 2018, 2018 4th International Conference on Electrical Engineering and Information & Communication Technology (iCEEiCT).

[19]  Anthony S. Maida,et al.  Multi-layer unsupervised learning in a spiking convolutional neural network , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[20]  Tatiana Baidyk,et al.  Improved method of handwritten digit recognition tested on MNIST database , 2004, Image Vis. Comput..

[21]  Gunhee Kim,et al.  Retrieval of Sentence Sequences for an Image Stream via Coherence Recurrent Convolutional Networks , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[23]  Luca Maria Gambardella,et al.  Flexible, High Performance Convolutional Neural Networks for Image Classification , 2011, IJCAI.

[24]  Kaushik Gobindram Pasi,et al.  Effect of parameter variations on accuracy of Convolutional Neural Network , 2016, 2016 International Conference on Computing, Analytics and Security Trends (CAST).