Cascade marker removal algorithm for thyroid ultrasound images

During thyroid ultrasound diagnosis, radiologists add markers such as pluses or crosses near a nodule’s edge to indicate the location of a nodule. For computer-aided detection, deep learning models achieve classification, segmentation, and detection by learning the thyroid’s texture in ultrasound images. Experiments show that manual markers are strong prior knowledge for data-driven deep learning models, which interferes with the judgment mechanism of computer-aided detection systems. Aiming at this problem, this paper proposes cascade marker removal algorithm for thyroid ultrasound images to eliminate the interference of manual markers. The algorithm consists of three parts. First, in order to highlight marked features, the algorithm extracts salient features in thyroid ultrasound images through feature extraction module. Secondly, mask correction module eliminates the interference of other features besides markers’ features. Finally, the marker removal module removes markers without destroying the semantic information in thyroid ultrasound images. Experiments show that our algorithm enables classification, segmentation, and object detection models to focus on the learning of pathological tissue features. At the same time, compared with mainstream image inpainting algorithms, our algorithm shows better performance on thyroid ultrasound images. In summary, our algorithm is of great significance for improving the stability and performance of computer-aided detection systems. Graphical Abstract During thyroid ultrasound diagnosis, doctors add markers such as pluses or crosses near nodule's edge to indicate the location of nodule. Manual markers are strong prior knowledge for data-driven deep learning models, which interferes the judgment mechanism of computer-aided diagnosis system based on deep learning. Markers make models overfit the specific labeling forms easily, and performs poorly on unmarked thyroid ultrasound images. Aiming at this problem, this paper proposes a cascade marker removal algorithm to eliminate the interference of manual markers. Our algorithm make deep learning models pay attention on nodules’ features of thyroid ultrasound images, which make computer-aided diagnosis system performs good in both marked imaging and unmarked imaging. During thyroid ultrasound diagnosis, doctors add markers such as pluses or crosses near nodule's edge to indicate the location of nodule. Manual markers are strong prior knowledge for data-driven deep learning models, which interferes the judgment mechanism of computer-aided diagnosis system based on deep learning. Markers make models overfit the specific labeling forms easily, and performs poorly on unmarked thyroid ultrasound images. Aiming at this problem, this paper proposes a cascade marker removal algorithm to eliminate the interference of manual markers. Our algorithm make deep learning models pay attention on nodules’ features of thyroid ultrasound images, which make computer-aided diagnosis system performs good in both marked imaging and unmarked imaging.

[1]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[2]  William D Middleton,et al.  ACR Thyroid Imaging, Reporting and Data System (TI-RADS): White Paper of the ACR TI-RADS Committee. , 2018, Journal of the American College of Radiology : JACR.

[3]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[4]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Matthias Bethge,et al.  ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness , 2018, ICLR.

[6]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  E. A. Gaston,et al.  The significance of nontoxic thyroid nodules. Final report of a 15-year study of the incidence of thyroid malignancy. , 1968, Annals of internal medicine.

[8]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[9]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Adam Finkelstein,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, SIGGRAPH 2009.

[11]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[13]  Jianguo Zhang,et al.  The PASCAL Visual Object Classes Challenge , 2006 .

[14]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  S. Mandel,et al.  A 64-year-old woman with a thyroid nodule. , 2004, JAMA.

[16]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[18]  William T. Freeman,et al.  On the Effectiveness of Visible Watermarks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Luc Van Gool,et al.  Efficient Non-Maximum Suppression , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[20]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Neil A. Dodgson,et al.  Proceedings Ninth IEEE International Conference on Computer Vision , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[22]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  L. Hegedüs,et al.  The Thyroid Nodule , 2004 .

[24]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[25]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[26]  Ting-Chun Wang,et al.  Image Inpainting for Irregular Holes Using Partial Convolutions , 2018, ECCV.

[27]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Xiaogang Wang,et al.  Residual Attention Network for Image Classification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Luc Van Gool,et al.  The 2005 PASCAL Visual Object Classes Challenge , 2005, MLCW.

[30]  D. Appleton,et al.  THE SPECTRUM OF THYROID DISEASE IN A COMMUNITY: THE WHICKHAM SURVEY , 1977, Clinical endocrinology.

[31]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Alexandru Telea,et al.  An Image Inpainting Technique Based on the Fast Marching Method , 2004, J. Graphics, GPU, & Game Tools.

[33]  William D Middleton,et al.  Thyroid Ultrasound Reporting Lexicon: White Paper of the ACR Thyroid Imaging, Reporting and Data System (TIRADS) Committee. , 2015, Journal of the American College of Radiology : JACR.

[34]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[35]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Hiroshi Ishikawa,et al.  Globally and locally consistent image completion , 2017, ACM Trans. Graph..

[37]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.