A Feedback Attention Network for Classification and Visualization of Thyroid Nodules on Ultrasound Images

With the maturity of AI technology, computer-aided diagnosis technology is gradually developing, even surpassing the performance of doctors. However, due to the lack of interpretability of deep learning, we can’t understand what the model has learned, and even determine whether the diagnostic model is credible. In this paper, Feedback Attention Network(FAN) is proposed, which visualizes the lesion area that plays a decisive role in precise prediction while classifying benign and malignant thyroid ultrasound images. The degree of fineness is adjustable, which is more refined than existing visualization methods. In addition, this paper uses the attention to make the model more sensitive to the lesion area and inhibit the influence of the non-lesion area. The experimental results show that the accuracy of benign and malignant classification reaches 90.76%±2.8%. Compared with the state-of-the-art classification networks, FAN has higher accuracy and fewer parameters. Above all, the visualization results show that FAN does learn the true nodule features, especially those at the edge of nodules, which proves FAN a credible aided diagnosis model.

[1]  Dahun Kim,et al.  Two-Phase Learning for Weakly Supervised Object Localization , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[2]  Daniel S. Kermany,et al.  Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning , 2018, Cell.

[3]  Tao Mei,et al.  Look Closer to See Better: Recurrent Attention Convolutional Neural Network for Fine-Grained Image Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Jinlian Ma,et al.  A pre‐trained convolutional neural network based method for thyroid nodule diagnosis , 2017, Ultrasonics.

[5]  Kai Liu,et al.  Thyroid Nodule Segmentation in Ultrasound Images Based on Cascaded Convolutional Neural Network , 2018, ICONIP.

[6]  Katherine W. Ferrara,et al.  Acoustical structured illumination for super-resolution ultrasound imaging , 2018, Communications Biology.

[7]  Yao Zhao,et al.  Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  P. Lakhani,et al.  Deep Learning at Chest Radiography: Automated Classification of Pulmonary Tuberculosis by Using Convolutional Neural Networks. , 2017, Radiology.

[9]  Junzhou Huang,et al.  Subtype Cell Detection with an Accelerated Deep Convolution Neural Network , 2016, MICCAI.

[10]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Andrew Y. Ng,et al.  CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning , 2017, ArXiv.

[12]  S. Thrun,et al.  Corrigendum: Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[13]  Aimin Hao,et al.  Learning from Weakly-Labeled Clinical Data for Automatic Thyroid Nodule Classification in Ultrasound Images , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[14]  A. Babajani-Feremi,et al.  Application of advanced machine learning methods on resting-state fMRI network for identification of mild cognitive impairment and Alzheimer’s disease , 2015, Brain Imaging and Behavior.

[15]  Maria Fazio,et al.  An Innovative Methodology for Big Data Visualization for Telemedicine , 2019, IEEE Transactions on Industrial Informatics.

[16]  Eun-Kyung Kim,et al.  Deep convolutional neural network for the diagnosis of thyroid nodules on ultrasound , 2019, Head & neck.

[17]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[18]  Yunchao Wei,et al.  Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi-Supervised Semantic Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[19]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[20]  Qiang Chen,et al.  Network In Network , 2013, ICLR.

[21]  Fa Wu,et al.  Cascade convolutional neural networks for automatic detection of thyroid nodules in ultrasound images , 2017, Medical physics.

[22]  Yong Jae Lee,et al.  Hide-and-Seek: Forcing a Network to be Meticulous for Weakly-Supervised Object and Action Localization , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[23]  Wenyu Liu,et al.  Weakly-Supervised Semantic Segmentation Network with Deep Seeded Region Growing , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[24]  Liang Chen,et al.  Attention-Gated Networks for Improving Ultrasound Scan Plane Detection , 2018, MICCAI 2018.

[25]  Laurence T. Yang,et al.  An Efficient Deep Learning Model to Predict Cloud Workload for Industry Informatics , 2018, IEEE Transactions on Industrial Informatics.

[26]  Eric Krokos,et al.  Deep-Learning-Assisted Volume Visualization , 2019, IEEE Transactions on Visualization and Computer Graphics.

[27]  Yun Fu,et al.  Tell Me Where to Look: Guided Attention Inference Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[28]  Hayit Greenspan,et al.  GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification , 2018, Neurocomputing.