Multi-scale Convolutional Neural Network Based on 3D Context Fusion for Lesion Detection

Lesion detection is an essential technique in medical diagnostic systems. Since there are great differences in intensity and appearance within a same lesion category, lesion detection from computed tomography (CT) scans is still a challenging task. Sufficiently using 3D context information become the research hotpot in lesion detection area, since algorithms can benefit from geometry and texture of lesions. Motivated by this trend, we propose a multi-scale CNN based on 3D context fusion, called M3DCF, for extracting lesion area from CT scans. In order to speed up the algorithm, the one-stage regression-based detector, rather than region proposal network, is adopted. Specifically, we employ 3D context fusion strategy that allows M3DCF fusing features from neighboring slices. Finally, we use a multi-scale scheme to combine low-level and high-level features. This strategy allows us to get more meaningful semantic information. The experimental results conducted on DeepLesion dataset indicates that the proposed method outperformed state-of-the-arts, including RetinaNet, Faster R-CNN, and 3DCE. The source code is available on https://github.com/JMUAIA/M3DCF.

[1]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  N. Levinson.  The Wiener (Root Mean Square) Error Criterion in Filter Design and Prediction , 1946 .

[3]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[5]  Shie Mannor,et al.  A Tutorial on the Cross-Entropy Method , 2005, Ann. Oper. Res..

[6]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[7]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[8]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[9]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Ran El-Yaniv,et al.  Binarized Neural Networks , 2016, NIPS.

[11]  Hyo-Eun Kim,et al.  Self-Transfer Learning for Weakly Supervised Lesion Localization , 2016, MICCAI.

[12]  Hao Chen,et al.  Automatic Detection of Cerebral Microbleeds From MR Images via 3D Convolutional Neural Networks , 2016, IEEE Transactions on Medical Imaging.

[13]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[14]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Ronald M. Summers,et al.  3D Context Enhanced Region-based Convolutional Neural Network for End-to-End Lesion Detection , 2018, MICCAI.

[16]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[17]  Koen E. A. van de Sande,et al.  Selective Search for Object Recognition , 2013, International Journal of Computer Vision.

[18]  J. L. Hock,et al.  An exact recursion for the composite nearest‐neighbor degeneracy for a 2×N lattice space , 1984 .

[19]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[21]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[22]  C. Willmott,et al.  Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance , 2005 .

[23]  Hiroshi Fujita,et al.  Automated detection of pulmonary nodules in PET/CT images: Ensemble false-positive reduction using a convolutional neural network technique. , 2016, Medical physics.

[24]  David Malah,et al.  Speech enhancement using a minimum mean-square error log-spectral amplitude estimator , 1984, IEEE Trans. Acoust. Speech Signal Process..

[25]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2015, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Le Lu,et al.  DeepLesion: automated mining of large-scale lesion annotations and universal lesion detection with deep learning , 2018, Journal of medical imaging.

[27]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.