Integrate domain knowledge in training multi-task cascade deep learning model for benign-malignant thyroid nodule classification on ultrasound images

Abstract The automatic and accurate diagnosis of thyroid nodules in ultrasound images is of great significance to reduce the workload and radiologists’ misdiagnosis rate. Although deep learning has shown strong image classification performance, the inherent limitations of medical images small dataset and time-consuming access to lesion annotations, leaving this work still facing challenges. In our study, a multi-task cascade deep learning model (MCDLM) was proposed, which integrates radiologists’ various domain knowledge (DK) and uses multimodal ultrasound images for automatic diagnosis of thyroid nodules. Specifically, we transfer the knowledge learned by U-net from the source domain to the target domain under the guidance of radiologist’ marks to obtain more accurate nodules’ segmentation results. We then quantify the nodules’ ultrasound features (UF) as conditions to assist the dual-path semi-supervised conditional generative adversarial network (DScGAN) in generating higher quality images obtaining more powerful discriminators. After that, we concatenate DScGAN learning’s image representation to train a supervised support vector machine (S3VM) for thyroid nodule classification. The experiment results on ultrasound images of 1030 patients suggest that the MCDLM model can achieve almost the same classification performance as the fully supervised learning (an accuracy of 90.01% and an AUC of 91.07%) using only about 35% of the full labeled dataset, which saves a lot of time and effort compared to traditional methods.