TDP: Two-dimensional perceptron for image recognition

Abstract Convolutional neural network (CNN) is widely applied to different areas due to good recognition performance. However, convolution operation is a complex computation and consumes the bulk of processing time for CNN. It is still a hot problem how to develop a novel model with good recognition performance for deep learning. Here, we propose a novel model, namely, two-dimensional perceptron (TDP), to get direct input of two-dimensional data for further processing. A TDP has a new network architecture and an innovative computation process of hidden neurons. In cases with the same number of hidden neurons, compared with multilayer perceptron (MLP), TDP achieves good recognition performance with 1 × -36 × speedup and a decrease of parameters by exceeding 97% on MNIST and COIL-20 datasets. Meanwhile, TDP obtains 1%–32% improvement of recognition accuracy in comparison to CNN on CIFAR-10 and SVHN datasets. Furthermore, on INFUSE dataset, TDP has an increase of F1 score by up to almost 11% in comparison with MLP and CNN. The results indicate that TDP is a promising and novel model with excellent recognition performance.

[1]  Pilsung Kang,et al.  Sentiment Classification with Word Attention based on Weakly Supervised Learning with a Convolutional Neural Network , 2017, ArXiv.

[2]  Andrew Y. Ng,et al.  Reading Digits in Natural Images with Unsupervised Feature Learning , 2011 .

[3]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[4]  Shaoguo Cui,et al.  Brain Tumor Automatic Segmentation Using Fully Convolutional Networks , 2017 .

[5]  Geoffrey E. Hinton,et al.  Matrix capsules with EM routing , 2018, ICLR.

[6]  Saman Haratizadeh,et al.  A hybrid supervised semi-supervised graph-based model to predict one-day ahead movement of global stock markets and commodity prices , 2018, Expert Syst. Appl..

[7]  Kaare Brandt Petersen,et al.  The Matrix Cookbook , 2006 .

[8]  Vishnu Naresh Boddeti,et al.  Local Binary Convolutional Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Reyer Zwiggelaar,et al.  Automated Breast Ultrasound Lesions Detection Using Convolutional Neural Networks , 2018, IEEE Journal of Biomedical and Health Informatics.

[11]  Michael Elad,et al.  Multilayer Convolutional Sparse Modeling: Pursuit and Dictionary Learning , 2017, IEEE Transactions on Signal Processing.

[12]  Geoffrey E. Hinton,et al.  Dynamic Routing Between Capsules , 2017, NIPS.

[13]  Sule Yildirim Yayilgan,et al.  The impact of deep learning on document classification using semantically rich representations , 2019, Inf. Process. Manag..

[14]  Leonardo Nogueira Matos,et al.  Deep Neural Networks for Acoustic Modeling in the Presence of Noise , 2018, IEEE Latin America Transactions.

[15]  Tsuyoshi Usagawa,et al.  Contextual keyword spotting in lecture video with deep convolutional neural network , 2017, 2017 International Conference on Advanced Computer Science and Information Systems (ICACSIS).