R-CapsNet: An Improvement of Capsule Network for More Complex Data

Convolutional neural networks (CNNs) have achieved the best performance in some fields. However, they still have some defects. CNNs need a lot of images for training; they will lose much information in the pooling layer, which reduces the spatial resolution. Facing such problems, Hinton et al. proposed a capsule network (CapsNet). Although the CapsNet has achieved the best accuracy on MNIST dataset, it has not performed well on Fashion-MNIST, Cifar-10 and other datasets. Naturally, we established an improved version of capsule network (R-CapsNet). Results have shown that when using R-CapsNet model, the loss gets decreased and the accuracy gets improved on FashionMNIST. In the meanwhile, the training parameters are reduced by nearly half. Specifically, it reduces by 4.5M. Comparisons show that our proposed model reports improved accuracy of around 0.56% over the existing state-of-the-art systems in literature. The test accuracy of R-CapsNet model is 1.32% higher than that of the original model. Furthermore, better results have been achieved on Cifar-10 with R-CapsNet model and it has easily increased by 10% compared to CapsNet.

[2]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[3]  N. Arunkumar,et al.  Convolutional neural network for bio-medical image segmentation with hardware acceleration , 2018, Cognitive Systems Research.

[4]  A. Çapar,et al.  License Plate Recognition From Still Images and Video Sequences: A Survey , 2008, IEEE Transactions on Intelligent Transportation Systems.

[5]  Geoffrey E. Hinton,et al.  Transforming Autoencoders , 2011 .

[6]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[7]  Geoffrey E. Hinton,et al.  Dynamic Routing Between Capsules , 2017, NIPS.

[8]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[9]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[10]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Marios Anthimopoulos,et al.  Lung Pattern Classification for Interstitial Lung Diseases Using a Deep Convolutional Neural Network , 2016, IEEE Transactions on Medical Imaging.

[13]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[14]  Chunheng Wang,et al.  Deep nonlinear metric learning with independent subspace analysis for face verification , 2012, ACM Multimedia.

[15]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Jianzhong Wu,et al.  Stacked Sparse Autoencoder (SSAE) for Nuclei Detection on Breast Cancer Histopathology Images , 2016, IEEE Transactions on Medical Imaging.

[17]  Chao Fang,et al.  Improving Protein Gamma-Turn Prediction Using Inception Capsule Networks , 2018, Scientific Reports.

[18]  Emmanuel Dufourq,et al.  EDEN: Evolutionary deep networks for efficient machine learning , 2017, 2017 Pattern Recognition Association of South Africa and Robotics and Mechatronics (PRASA-RobMech).

[19]  Ali Miri,et al.  Using the Extreme Learning Machine (ELM) technique for heart disease diagnosis , 2015, 2015 IEEE Canada International Humanitarian Technology Conference (IHTC2015).

[20]  Maheshkumar H. Kolekar,et al.  Classification of fashion article images using convolutional neural networks , 2017, 2017 Fourth International Conference on Image Information Processing (ICIIP).

[21]  Roland Vollgraf,et al.  Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms , 2017, ArXiv.

[22]  Huijun Gao,et al.  A convolutional neural network based on a capsule network with strong generalization for bearing fault diagnosis , 2019, Neurocomputing.

[23]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[24]  Kyung-shik Shin,et al.  Hierarchical convolutional neural networks for fashion image classification , 2019, Expert Syst. Appl..