Parameter Transfer Deep Neural Network for Single-Modal B-Mode Ultrasound-Based Computer-Aided Diagnosis

Elastography ultrasound (EUS) imaging has shown its effectiveness for diagnosis of tumors by providing additional information about tissue stiffness to the conventional B-mode ultrasound (BUS). However, due to the lack of EUS devices and experienced sonologists, EUS is not widely used, especially in rural areas. It is still a challenging task to improve the performance of the single-modal BUS-based computer-aided diagnosis (CAD) for tumors. In this work, we propose a novel transfer learning (TL)–based deep neural network (DNN) algorithm, named CW-PM-DNN, for the BUS-based CAD by transferring diagnosis knowledge from EUS during model training. CW-PM-DNN integrates both the feature-level and classifier-level knowledge transfer into a unified framework. In the feature-level TL, a bichannel DNN is learned by the cross-weight-based multimodal DL (MDL-CW) algorithm to transfer informative features from EUS to BUS. In the classifier-level TL, a projective model (PM)–based classifier is then embedded to the pretrained bichannel DNN to implement the parameter transfer in the classifier model at the second stage. The back-propagation procedure is then applied to optimize the whole CW-PM-DNN to further improve its performance. Experimental results on two bimodal ultrasound tumor datasets demonstrate that the proposed CW-PM-DNN achieves the best classification accuracy, sensitivity, and specificity of 89.02 ± 1.54%, 88.37 ± 4.72%, and 89.63 ± 4.06%, respectively, for the breast ultrasound dataset, and the corresponding values of 80.57 ± 3.41%, 76.67 ± 3.85%, and 83.94 ± 3.95%, respectively, for the prostate ultrasound dataset. The proposed two-stage TL-based CW-PM-DNN algorithm outperforms all the compared algorithms. It is also proved that the performance of the BUS-based CAD can be significantly improved by transferring the knowledge of EUS. It suggests that CW-PM-DNN has the potential for more applications in the field of medical image–based CAD.

[1]  Xuelong Li,et al.  Segmentation of breast ultrasound image with semantic classification of superpixels , 2020, Medical Image Anal..

[2]  Jie Zhu,et al.  Shearlet-based texture feature extraction for classification of breast tumor in ultrasound image , 2013, Biomed. Signal Process. Control..

[3]  Michael I. Jordan,et al.  Learning Transferable Features with Deep Adaptation Networks , 2015, ICML.

[4]  Jeff A. Bilmes,et al.  Deep Canonical Correlation Analysis , 2013, ICML.

[5]  Yongli Guo,et al.  Erratum to “The Feasibility of Xpert MTB/RIF Testing to Detect Rifampicin Resistance among Childhood Tuberculosis for Prevalence Surveys in Northern China” , 2017, BioMed research international.

[6]  Qi Zhang,et al.  Sonoelastomics for Breast Tumor Classification: A Radiomics Approach with Clustering-Based Feature Selection on Sonoelastography. , 2017, Ultrasound in medicine & biology.

[7]  Hamid R. Rabiee,et al.  MDL-CW: A Multimodal Deep Learning Framework with CrossWeights , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Shuang Song,et al.  Dual-mode artificially-intelligent diagnosis of breast tumours in shear-wave elastography and B-mode ultrasound using deep polynomial networks. , 2019, Medical engineering & physics.

[9]  Wenzhong Guo,et al.  Deep Multimodal Representation Learning: A Survey , 2019, IEEE Access.

[10]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[11]  Guang-Bin Huang,et al.  Trends in extreme learning machines: A review , 2015, Neural Networks.

[12]  F. Alam,et al.  Bimodal Multiparameter-Based Approach for Benign-Malignant Classification of Breast Tumors. , 2015, Ultrasound in medicine & biology.

[13]  Graham W. Taylor,et al.  Deep Multimodal Learning: A Survey on Recent Advances and Trends , 2017, IEEE Signal Processing Magazine.

[14]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[15]  Bo Peng,et al.  Cascaded Multi-Column RVFL+ Classifier for Single-Modal Neuroimaging-Based Diagnosis of Parkinson's Disease , 2019, IEEE Transactions on Biomedical Engineering.

[16]  Dacheng Tao,et al.  On Combining Biclustering Mining and AdaBoost for Breast Tumor Classification , 2020, IEEE Transactions on Knowledge and Data Engineering.

[17]  NahavandiSaeid,et al.  Extreme learning machine based transfer learning algorithms , 2017 .

[18]  Chao Chen,et al.  Parameter Transfer Extreme Learning Machine based on Projective Model , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[19]  Qi Zhang,et al.  Deep learning based classification of breast tumors with shear-wave elastography. , 2016, Ultrasonics.

[20]  M. Chammas,et al.  Ultrasound Elastography: Review of Techniques and Clinical Applications , 2017, Theranostics.

[21]  Andre Esteva,et al.  A guide to deep learning in healthcare , 2019, Nature Medicine.

[22]  David Zhang,et al.  Robust Visual Knowledge Transfer via Extreme Learning Machine-Based Domain Adaptation , 2016, IEEE Transactions on Image Processing.

[23]  Jun Shi,et al.  Learning using privileged information improves neuroimaging-based CAD of Alzheimer’s disease: a comparative study , 2019, Medical & Biological Engineering & Computing.

[24]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[25]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[26]  Chao Yang,et al.  A Survey on Deep Transfer Learning , 2018, ICANN.

[27]  Peng-Bo Zhang,et al.  A New Learning Paradigm for Random Vector Functional-Link Network: RVFL+ , 2017, Neural Networks.

[28]  Barbara Caputo,et al.  Learning Categories From Few Examples With Multi Model Knowledge Transfer , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Honglak Lee,et al.  Improved Multimodal Deep Learning with Variation of Information , 2014, NIPS.

[30]  Taghi M. Khoshgoftaar,et al.  A survey of transfer learning , 2016, Journal of Big Data.

[31]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Ilias Gatos,et al.  A new computer aided diagnosis system for evaluation of chronic liver disease with ultrasound shear wave elastography imaging. , 2016, Medical physics.

[33]  SchmidhuberJürgen Deep learning in neural networks , 2015 .

[34]  Qinghua Huang,et al.  Machine Learning in Ultrasound Computer-Aided Diagnostic Systems: A Survey , 2018, BioMed research international.

[35]  Chris H. Q. Ding,et al.  R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization , 2006, ICML.

[36]  Ivor W. Tsang,et al.  Learning With Augmented Features for Supervised and Semi-Supervised Heterogeneous Domain Adaptation , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Hong Zhou,et al.  Ultrasound-based differentiation of malignant and benign thyroid Nodules: An extreme learning machine approach , 2017, Comput. Methods Programs Biomed..

[38]  Dong Ni,et al.  Deep Learning in Medical Ultrasound Analysis: A Review , 2019, Engineering.

[39]  Josien P. W. Pluim,et al.  Not‐so‐supervised: A survey of semi‐supervised, multi‐instance, and transfer learning in medical image analysis , 2018, Medical Image Anal..

[40]  Guang-Bin Huang,et al.  Extreme Learning Machine for Multilayer Perceptron , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[41]  Alejandro F. Frangi,et al.  Classification of breast lesions in ultrasonography using sparse logistic regression and morphology‐based texture features , 2018, Medical physics.

[42]  Shihui Ying,et al.  Quaternion Grassmann average network for learning representation of histopathological image , 2019, Pattern Recognit..

[43]  Roi Livni,et al.  An Algorithm for Training Polynomial Networks , 2013, 1304.7045.

[44]  Eliseo Guallar,et al.  Diagnostic accuracy and reliability of ultrasonography for the detection of fatty liver: A meta‐analysis , 2011, Hepatology.

[45]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[46]  Fan Zhang,et al.  Evolutionary optimized fuzzy reasoning with mined diagnostic patterns for classification of breast tumors in ultrasound , 2019, Inf. Sci..

[47]  Ferdinand Frauscher,et al.  Ultrasound of the prostate , 2010, Cancer imaging : the official publication of the International Cancer Imaging Society.

[48]  Saeid Nahavandi,et al.  Extreme learning machine based transfer learning algorithms: A survey , 2017, Neurocomputing.