An improved CapsNet applied to recognition of 3D vertebral images

Deep learning is currently widely applied in medical image processing and has achieved good results. However, recognizing vertebrae via image processing remains a challenging problem due to their complex spatial structures. CapsNet is a newly proposed network whose characteristics compensate for some shortcomings of traditional CNNs, and it has been shown to perform well on many tasks, including medical image recognition. In this paper, we applied a modified CapsNet to recognise 3D vertebral images by introducing an RNN module into CapsNet to further enhance its learning ability. This new network is called RNNinCaps, and it achieves the highest recognition performance on 3D vertebral images (the average accuracy of RNNinCaps exceeds the accuracy of the original CapsNet by 46.2% and that of a traditional CNN by 12.6%). RNNinCaps also performs better than several mainstream networks. RNNinCaps can promotes CapsNet’s application in the field of 3D medical image recognition.

[1]  Sotirios A. Tsaftaris,et al.  Medical Image Computing and Computer Assisted Intervention , 2017 .

[2]  Lorenzo Torresani,et al.  Learning Spatiotemporal Features with 3D Convolutional Networks , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[3]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[4]  Geoffrey E. Hinton,et al.  Matrix capsules with EM routing , 2018, ICLR.

[5]  Nasir M. Rajpoot,et al.  A Spatially Constrained Deep Learning Framework for Detection of Epithelial Tumor Nuclei in Cancer Histology Images , 2015, Patch-MI@MICCAI.

[6]  Jiebo Luo,et al.  Joint Vertebrae Identification and Localization in Spinal CT Images by Combining Short- and Long-Range Contextual Information , 2018, IEEE Transactions on Medical Imaging.

[7]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[8]  Dit-Yan Yeung,et al.  Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting , 2015, NIPS.

[9]  Joseph E. Burns.  Imaging of the Spine: A Medical and Physical Perspective , 2015 .

[10]  Andrew W. Senior,et al.  Long Short-Term Memory Based Recurrent Neural Network Architectures for Large Vocabulary Speech Recognition , 2014, ArXiv.

[11]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Yaohui Jin,et al.  MCapsNet: Capsule Network for Text with Multi-Task Learning , 2018, EMNLP.

[13]  Shadi Albarqouni,et al.  Capsule Networks against Medical Imaging Data Challenges , 2018, CVII-STENT/LABELS@MICCAI.

[14]  Konstantinos N. Plataniotis,et al.  Brain Tumor Type Classification via Capsule Networks , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[15]  Yang Jin,et al.  Capsule Network Performance on Complex Data , 2017, ArXiv.

[16]  Hien Van Nguyen,et al.  Fast CapsNet for Lung Cancer Screening , 2018, MICCAI.

[17]  Geoffrey E. Hinton,et al.  Dynamic Routing Between Capsules , 2017, NIPS.

[18]  Amara Dinesh Kumar,et al.  Novel Deep Learning Model for Traffic Sign Detection Using Capsule Networks , 2018, ArXiv.

[19]  Andrew Y. Ng,et al.  CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning , 2017, ArXiv.

[20]  Purang Abolmaesumi,et al.  Fast Automatic Vertebrae Detection and Localization in Pathological CT Scans - A Deep Learning Approach , 2015, MICCAI.

[21]  Yong Fan,et al.  A deep learning model integrating FCNNs and CRFs for brain tumor segmentation , 2017, Medical Image Anal..

[22]  Su-Lin Lee,et al.  Intravascular Imaging and Computer Assisted Stenting, and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis , 2017, Lecture Notes in Computer Science.

[23]  Andrew H. Beck,et al.  Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer , 2017, JAMA.

[24]  Purang Abolmaesumi,et al.  Deep learning for automatic localization, identification, and segmentation of vertebral bodies in volumetric MR images , 2015, Medical Imaging.

[25]  V. P. Gladis Pushpa Rathi,et al.  Brain Tumor Detection and Classification Using Deep Learning Classifier on MRI Images , 2015 .

[26]  Chenming Xu,et al.  An Attention-based BI-GRU-CapsNet Model for Hypernymy Detection between Compound Entities , 2018, 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[27]  Hao Chen,et al.  Automatic Localization and Identification of Vertebrae in Spine CT via a Joint Learning Model with Deep Neural Networks , 2015, MICCAI.

[28]  Qi Zhang,et al.  Deep learning based classification of breast tumors with shear-wave elastography. , 2016, Ultrasonics.