Computer-aided diagnosis of breast ultrasound images using ensemble learning from convolutional neural networks

Breast ultrasound and computer aided diagnosis (CAD) has been used to classify tumors into benignancy or malignancy. However, conventional CAD software has some problems (such as handcrafted features are hard to design; conventional CAD systems are difficult to confirm overfitting problems, etc.). In our study, we propose a CAD system for tumor diagnosis using an image fusion method combined with different image content representations and ensemble different CNN architectures on US images. The CNN-based method proposed in this study includes VGGNet, ResNet, and DenseNet. In our private dataset, there was a total of 1687 tumors that including 953 benign and 734 malignant tumors. The accuracy, sensitivity, specificity, precision, F1 score and the AUC of the proposed method were 91.10%, 85.14%, 95.77%, 94.03%, 89.36%, and 0.9697 respectively. In the open dataset (BUSI), there was a total of 697 tumors that including 437 benign lesions, 210 malignant tumors, and 133 normal images. The accuracy, sensitivity, specificity, precision, F1 score, and the AUC of the proposed method were 94.62%, 92.31%, 95.60%, 90%, 91.14%, and 0.9711. In conclusion, the results indicated different image content representations that affect the prediction performance of the CAD system, more image information improves the prediction performance, and the tumor shape feature can improve the diagnostic effect.

[1]  Sepp Hochreiter,et al.  The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions , 1998, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[2]  Manu Goyal,et al.  End-to-end breast ultrasound lesions recognition with a deep learning approach , 2018, Medical Imaging.

[3]  W. Moon,et al.  Ultrasound breast tumor image computer-aided diagnosis with texture and morphological features. , 2008, Academic radiology.

[4]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[5]  Cristina V. Lopes,et al.  Bagging gradient-boosted trees for high precision, low variance ranking models , 2011, SIGIR.

[6]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Ruey-Feng Chang,et al.  Computer-aided prediction of axillary lymph node status in breast cancer using tumor surrounding tissue features in ultrasound images , 2017, Comput. Methods Programs Biomed..

[8]  Bin-Da Liu,et al.  Breast Tumor Classification of Ultrasound Images Using Wavelet-Based Channel Energy and ImageJ , 2009, IEEE Journal of Selected Topics in Signal Processing.

[9]  Wenjian Qin,et al.  Comparison of Transferred Deep Neural Networks in Ultrasonic Breast Masses Discrimination , 2018, BioMed research international.

[10]  Nico Karssemeijer,et al.  Computer-Aided Lesion Diagnosis in Automated 3-D Breast Ultrasound Using Coronal Spiculation , 2012, IEEE Transactions on Medical Imaging.

[11]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[12]  Wei Tang,et al.  Ensembling neural networks: Many could be better than all , 2002, Artif. Intell..

[13]  Rangaraj M. Rangayyan,et al.  Measures of acutance and shape for classification of breast tumors , 1997, IEEE Transactions on Medical Imaging.

[14]  Michael Galperin,et al.  Breast mass classification in sonography with transfer learning using a deep convolutional neural network and color conversion , 2019, Medical physics.

[15]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[17]  Lina Arbash Meinel,et al.  Breast MRI lesion classification: Improved performance of human readers with a backpropagation neural network computer‐aided diagnosis (CAD) system , 2007, Journal of magnetic resonance imaging : JMRI.

[18]  Reyer Zwiggelaar,et al.  Automated Breast Ultrasound Lesions Detection Using Convolutional Neural Networks , 2018, IEEE Journal of Biomedical and Health Informatics.

[19]  Ulrike M. Hamper,et al.  Breast Ultrasound: A Systematic Approach to Technique and Image Interpretation , 1999 .

[20]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Lubomir M. Hadjiiski,et al.  Malignant and benign breast masses on 3D US volumetric images: effect of computer-aided diagnosis on radiologist accuracy. , 2007, Radiology.

[22]  Yin-Yin Liao,et al.  Classification of Benign and Malignant Breast Tumors by 2-D Analysis Based on Contour Description and Scatterer Characterization , 2010, IEEE Transactions on Medical Imaging.

[23]  Shuiwang Ji,et al.  Deep convolutional neural networks for multi-modality isointense infant brain image segmentation , 2015, NeuroImage.

[24]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[25]  Ronald M. Summers,et al.  Improving Computer-Aided Detection Using Convolutional Neural Networks and Random View Aggregation , 2015, IEEE Transactions on Medical Imaging.

[26]  A. Stavros,et al.  Solid breast nodules: use of sonography to distinguish between benign and malignant lesions. , 1995, Radiology.

[27]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[28]  Seokmin Han,et al.  A deep learning framework for supporting the classification of breast lesions in ultrasound images , 2017, Physics in medicine and biology.

[29]  Juan Shan,et al.  Computer-Aided Diagnosis for Breast Ultrasound Using Computerized BI-RADS Features and Machine Learning Methods. , 2016, Ultrasound in medicine & biology.

[30]  Xin Yao,et al.  Ensemble learning via negative correlation , 1999, Neural Networks.

[31]  Alexander Toet,et al.  Semi-hidden target recognition in gated viewer images fused with thermal IR images , 2014, Inf. Fusion.

[32]  Walid Al-Dhabyani,et al.  Dataset of breast ultrasound images , 2019, Data in brief.

[33]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[34]  Gui-Song Xia,et al.  Transferring Deep Convolutional Neural Networks for the Scene Classification of High-Resolution Remote Sensing Imagery , 2015, Remote. Sens..

[35]  B Huynh,et al.  MO-DE-207B-06: Computer-Aided Diagnosis of Breast Ultrasound Images Using Transfer Learning From Deep Convolutional Neural Networks. , 2016, Medical physics.

[36]  Ling Zhang,et al.  Automated breast cancer detection and classification using ultrasound images: A survey , 2015, Pattern Recognit..

[37]  Jitendra Malik,et al.  Region-Based Convolutional Networks for Accurate Object Detection and Segmentation , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Nico Karssemeijer,et al.  Using computer-aided detection in mammography as a decision support , 2010, European Radiology.

[39]  Xiaolan Tang,et al.  Breast Ultrasound Image Classification and Segmentation Using Convolutional Neural Networks , 2018, PCM.

[40]  W. Moon,et al.  Computer‐aided diagnosis using morphological features for classifying breast lesions on ultrasound , 2008, Ultrasound in obstetrics & gynecology : the official journal of the International Society of Ultrasound in Obstetrics and Gynecology.

[41]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[42]  Ruey-Feng Chang,et al.  Classification of breast ultrasound images using fractal feature. , 2005, Clinical imaging.

[43]  Zheng Liu,et al.  A novel image fusion framework for night-vision navigation and surveillance , 2015, Signal Image Video Process..

[44]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Yifeng Niu,et al.  Airborne Infrared and Visible Image Fusion for Target Perception Based on Target Region Segmentation and Discrete Wavelet Transform , 2012 .

[46]  Qi Zhang,et al.  Deep learning based classification of breast tumors with shear-wave elastography. , 2016, Ultrasonics.

[47]  N. Petrick,et al.  Improvement of radiologists' characterization of mammographic masses by using computer-aided diagnosis: an ROC study. , 1999, Radiology.

[48]  Bir Bhanu,et al.  Fusion of color and infrared video for moving human detection , 2007, Pattern Recognit..