Ultrasound-based dominant intraprostatic lesion classification with swin transformer

In prostate brachytherapy, focal boost on dominant intraprostatic lesions (DILs) can reduce the recurrence rate while keeping low toxicity. In recent years, ultrasound (US) prostate tissue characterization has demonstrated the feasibility in detecting dominant intraprostatic lesions. With recent developments in computer-aided diagnosis (CAD), deep learningbased methods have provided solutions for efficient analysis of US images. In this study, we aim to develop a Shiftedwindows (Swin) Transformer-based method for DIL classification. The self-attention layers in Swin Transformer allow efficient feature discrimination between benign tissues and intraprostatic lesions. We simplified the structure of Swin Transformer to avoid overfitting on a small dataset. The proposed transformer structure achieved 83% accuracy and 0.86 AUC at patient level on three-fold cross validation, demonstrating the feasibility of applying our method for dominant lesion classification from US images, which is of clinical significance for radiotherapy treatment planning.

[1]  Tian Liu,et al.  Reinforcement learning in medical image analysis: Concepts, applications, challenges, and future directions , 2022, Journal of applied clinical medical physics.

[2]  J. Roper,et al.  Abdomen CT Multi-organ Segmentation Using Token-based MLP-Mixer. , 2022, Medical physics.

[3]  J. Bradley,et al.  Validation of a deep learning-based material estimation model for Monte Carlo dose calculation in proton therapy , 2022, Physics in medicine and biology.

[4]  J. Roper,et al.  Male pelvic multi-organ segmentation using token-based transformer Vnet , 2022, Physics in Medicine and Biology.

[5]  Qian Wang,et al.  Dual-energy CT based mass density and relative stopping power estimation for proton therapy using physics-informed deep learning , 2022, Physics in medicine and biology.

[6]  A. Dosovitskiy,et al.  Do Vision Transformers See Like Convolutional Neural Networks? , 2021, NeurIPS.

[7]  W. Curran,et al.  A review of deep learning based methods for medical image multi-organ segmentation. , 2021, Physica medica : PM : an international journal devoted to the applications of physics to medicine and biology : official journal of the Italian Association of Biomedical Physics.

[8]  Grace J Gang,et al.  Generative Adversarial Networks and Radiomics Supervision for Lung Lesion Synthesis. , 2021, Proceedings of SPIE--the International Society for Optical Engineering.

[9]  Matthieu Cord,et al.  Training data-efficient image transformers & distillation through attention , 2020, ICML.

[10]  S. Gelly,et al.  An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , 2020, ICLR.

[11]  Stephen Lin,et al.  Swin Transformer: Hierarchical Vision Transformer using Shifted Windows , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[12]  Yang Lei,et al.  A review on medical imaging synthesis using deep learning and its clinical applications , 2020, Journal of applied clinical medical physics.

[13]  Sibo Tian,et al.  A planning study of focal dose escalations to multiparametric MRI-defined dominant intraprostatic lesions in prostate proton radiation therapy. , 2020, British Journal of Radiology.

[14]  W. Curran,et al.  Deep learning in medical image registration: a review , 2019, Physics in medicine and biology.

[15]  S. Gelly,et al.  Big Transfer (BiT): General Visual Representation Learning , 2019, ECCV.

[16]  Paulo Mazzoncini de Azevedo Marques,et al.  Artificial intelligence, machine learning, computer-aided diagnosis, and radiomics: advances in imaging towards to precision medicine , 2019, Radiologia brasileira.

[17]  Yang Lei,et al.  Multiparametric MRI-guided dose boost to dominant intraprostatic lesions in CT-based High-dose-rate prostate brachytherapy. , 2019, The British journal of radiology.

[18]  Larry S. Davis,et al.  An Analysis of Scale Invariance in Object Detection - SNIP , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[19]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[21]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[22]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.