Improving Weakly Supervised Lesion Segmentation using Multi-Task Learning

We introduce the concept of multi-task learning to weakly-supervised lesion segmentation, one of the most critical and challenging tasks in medical imaging. Due to the lesions’ heterogeneous nature, it is difficult for machine learning models to capture the corresponding variability. We propose to jointly train a lesion segmentation model and a lesion classifier in a multi-task learning fashion, where the supervision of the latter is obtained by clustering the RECIST measurements of the lesions. We evaluate our approach specifically on liver lesion segmentation and more generally on lesion segmentation in computed tomography (CT), as well as segmentation of skin lesions from dermatoscopic images. We show that the proposed joint training improves the quality of the lesion segmentation by 4% percent according to the Dice coefficient and 6% according to averaged Hausdorff distance (AVD), while reducing the training time required by up to 75%.

[1]  Su Ruan,et al.  Multi-task deep learning based CT imaging analysis for COVID-19 pneumonia: Classification and segmentation , 2020, Computers in Biology and Medicine.

[2]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[3]  Wonhee Lee,et al.  Multi-Task Self-Supervised Object Detection via Recycling of Bounding Box Annotations , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Youbao Tang,et al.  Weakly Supervised Lesion Co-Segmentation on Ct Scans , 2020, 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI).

[5]  Youbao Tang,et al.  Weakly-supervised lesion segmentation on CT scans using co-segmentation , 2020, Medical Imaging.

[6]  Allan Hanbury,et al.  Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool , 2015, BMC Medical Imaging.

[7]  Khaled Alsaih,et al.  Evaluation of Deep Neural Networks for Semantic Segmentation of Prostate in T2W MRI , 2020, Sensors.

[8]  Yu Zhang,et al.  A Survey on Multi-Task Learning , 2017, IEEE Transactions on Knowledge and Data Engineering.

[9]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[10]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[11]  Le Lu,et al.  DeepLesion: automated mining of large-scale lesion annotations and universal lesion detection with deep learning , 2018, Journal of medical imaging.

[12]  Yuan Gao,et al.  Weakly Supervised Deep Learning for COVID-19 Infection Detection and Classification From CT Images , 2020, IEEE Access.

[13]  Hao Chen,et al.  The Liver Tumor Segmentation Benchmark (LiTS) , 2019, Medical Image Anal..

[14]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[15]  Yassine Ruichek,et al.  Survey on semantic segmentation using deep learning techniques , 2019, Neurocomputing.

[16]  Chaoqun Wang,et al.  Pattern-Structure Diffusion for Multi-Task Learning , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Jitendra Malik,et al.  Which Tasks Should Be Learned Together in Multi-task Learning? , 2019, ICML.

[18]  Brian Kingsbury,et al.  New types of deep neural network learning for speech recognition and related applications: an overview , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[19]  Nicolas Thome,et al.  Multitask Classification and Segmentation for Cancer Diagnosis in Mammography , 2019, 1909.05397.

[20]  K. Brock,et al.  Automated Contouring of Contrast and Noncontrast Computed Tomography Liver Images With Fully Convolutional Networks , 2020, Advances in radiation oncology.

[21]  Kerstin Voigt,et al.  Stock Index Prediction with Multi-task Learning and Word Polarity Over Time , 2020, ArXiv.

[22]  Yu Liu,et al.  A review of semantic segmentation using deep neural networks , 2017, International Journal of Multimedia Information Retrieval.

[23]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Dong Yang,et al.  Going to Extremes: Weakly Supervised Medical Image Segmentation , 2020, Mach. Learn. Knowl. Extr..

[25]  Youbao Tang,et al.  Accurate Weakly-Supervised Deep Lesion Segmentation using Large-Scale Clinical Annotations: Slice-Propagated 3D Mask Generation from 2D RECIST , 2018, MICCAI.

[26]  Konstantinos Kamnitsas,et al.  DeepCut: Object Segmentation From Bounding Box Annotations Using Convolutional Neural Networks , 2016, IEEE Transactions on Medical Imaging.

[27]  Chunhua Shen,et al.  A Mutual Bootstrapping Model for Automated Skin Lesion Segmentation and Classification , 2020, IEEE Transactions on Medical Imaging.

[28]  Yi Su,et al.  A Novel Multi-task Deep Learning Model for Skin Lesion Segmentation and Classification , 2017, ArXiv.

[29]  Xiaodong Liu,et al.  Multi-Task Deep Neural Networks for Natural Language Understanding , 2019, ACL.

[30]  Nima Tajbakhsh,et al.  Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation , 2019, Medical Image Anal..

[31]  Harald Kittler,et al.  Descriptor : The HAM 10000 dataset , a large collection of multi-source dermatoscopic images of common pigmented skin lesions , 2018 .

[32]  Matthijs Douze,et al.  Deep Clustering for Unsupervised Learning of Visual Features , 2018, ECCV.

[33]  L. Schwartz,et al.  New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1). , 2009, European journal of cancer.