Systematic Clinical Evaluation of A Deep Learning Method for Medical Image Segmentation: Radiosurgery Application

We systematically evaluate a Deep Learning (DL) method in a 3D medical image segmentation task. Our segmentation method is integrated into the radiosurgery treatment process and directly impacts the clinical workflow. With our method, we address the relative drawbacks of manual segmentation: high inter-rater contouring variability and high time consumption of the contouring process. The main extension over the existing evaluations is the careful and detailed analysis that could be further generalized on other medical image segmentation tasks. Firstly, we analyze the changes in the inter-rater detection agreement. We show that the segmentation model reduces the ratio of detection disagreements from 0.162 to 0.085 (p < 0.05). Secondly, we show that the model improves the inter-rater contouring agreement from 0.845 to 0.871 surface Dice Score (p < 0.05). Thirdly, we show that the model accelerates the delineation process in between 1.6 and 2.0 times (p < 0.05). Finally, we design the setup of the clinical experiment to either exclude or estimate the evaluation biases, thus preserve the significance of the results. Besides the clinical evaluation, we also summarize the intuitions and practical ideas for building an efficient DL-based model for 3D medical image segmentation.

[1]  Steve B. Jiang,et al.  A deep convolutional neural network-based automatic delineation strategy for multiple brain metastases stereotactic radiosurgery , 2017, PloS one.

[2]  Brent van der Heyden,et al.  Evaluation of measures for assessing time-saving of automatic organ-at-risk segmentation in radiotherapy , 2019, Physics and imaging in radiation oncology.

[3]  Berkman Sahiner,et al.  Deep learning in medical imaging and radiation therapy. , 2018, Medical physics.

[4]  Elisabetta Binaghi,et al.  Investigating the Behaviour of Machine Learning Techniques to Segment Brain Metastases in Radiation Therapy Planning , 2019, Applied Sciences.

[5]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Paul Aljabar,et al.  Clinical evaluation of atlas and deep learning based automatic contouring for lung cancer. , 2017, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[7]  Szu-Yeu Hu,et al.  Multimodal Volume-Aware Detection and Segmentation for Brain Metastases Radiosurgery , 2019, AIRT@MICCAI.

[8]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[9]  Daniel L. Rubin,et al.  Deep learning enables automatic detection and segmentation of brain metastases on multisequence MRI , 2019, Journal of magnetic resonance imaging : JMRI.

[10]  June-Goo Lee,et al.  Deep Learning in Medical Imaging: General Overview , 2017, Korean journal of radiology.

[11]  Geraint Rees,et al.  Deep learning to achieve clinically applicable segmentation of head and neck anatomy for radiotherapy , 2018, ArXiv.

[12]  Odelin Charron,et al.  Automatic detection and segmentation of brain metastases on multimodal MR images with a deep convolutional neural network , 2018, Comput. Biol. Medicine.

[13]  N. McVicar,et al.  Comparing Deep Learning-based Auto-segmentation of Organs at Risk and Clinical Target Volumes to Expert Inter-Observer Variability in Radiotherapy Planning , 2019, International Journal of Radiation Oncology*Biology*Physics.

[14]  Anvar Kurmukov,et al.  Universal Loss Reweighting to Balance Lesion Size Inequality in 3D Medical Image Segmentation , 2020, MICCAI.

[15]  Jianrong Dai,et al.  Deep Learning Improved Clinical Target Volume Contouring Quality and Efficiency for Postoperative Radiation Therapy in Non-small Cell Lung Cancer , 2019, Front. Oncol..

[16]  H. Jokura,et al.  Multi-institutional study of the variability in target delineation for six targets commonly treated with radiosurgery , 2018, Acta oncologica.

[17]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[18]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[19]  A. Gladwish,et al.  Clinical evaluation of deep learning and atlas based auto-contouring of bladder and rectum for prostate radiotherapy. , 2020, Practical radiation oncology.

[20]  Klaus H. Maier-Hein,et al.  No New-Net , 2018, 1809.10483.

[21]  J. Cheng,et al.  Randomized multi-reader evaluation of automated detection and segmentation of brain tumors in stereotactic radiosurgery with deep neural networks , 2021, Neuro-oncology.

[22]  Hannu Laaksonen,et al.  Clinical evaluation of a full-image deep segmentation algorithm for the male pelvis on cone-beam CT and CT. , 2019, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[23]  M. Torrens,et al.  Standardization of terminology in stereotactic radiosurgery: Report from the Standardization Committee of the International Leksell Gamma Knife Society: special topic. , 2014, Journal of neurosurgery.

[24]  Egor Krivov,et al.  Deep Learning for Brain Tumor Segmentation in Radiosurgery: Prospective Clinical Evaluation , 2019, BrainLes@MICCAI.

[25]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[26]  C. Njeh,et al.  Tumor delineation: The weakest link in the search for accuracy in radiotherapy , 2008, Journal of medical physics.

[27]  Hans Meine,et al.  Reducing inter-observer variability and interaction time of MR liver volumetry by combining automatic CNN-based liver segmentation and manual corrections , 2019, PloS one.

[28]  et al.,et al.  Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge , 2018, ArXiv.

[29]  V. Visser-Vandewalle,et al.  Deep convolutional neural networks for automated segmentation of brain metastases trained on clinical data , 2020, Radiation Oncology.

[30]  Egor Krivov,et al.  Tumor Delineation For Brain Radiosurgery by a ConvNet and Non-Uniform Patch Generation , 2018, Patch-MI@MICCAI.

[31]  Egor Krivov,et al.  Accelerating 3D Medical Image Segmentation by Adaptive Small-Scale Target Localization , 2021, J. Imaging.

[32]  A. Beltramello,et al.  Clinical target volume delineation in glioblastomas: pre-operative versus post-operative/pre-radiotherapy MRI. , 2011, The British journal of radiology.

[33]  Alex Lallement,et al.  Survey on deep learning for radiotherapy , 2018, Comput. Biol. Medicine.