Volume-based Performance not Guaranteed by Promising Patch-based Results in Medical Imaging

Whole-body MRIs are commonly used to screen for early signs of cancer. In addi-tion to the small size of tumours at onset, variations in individuals, tumour types, and MRI machines increase the difficulty of finding tumours in these scans. Using patches, rather than whole-body scans, to train a deep-learning-based segmentation model with a custom compound patch loss function, several augmentations, and ad-ditional synthetically generated training data to identify areas where there is a high probability of a tumour provided promising results at the patch-level. However, applying the patch-based model to the entire volume did not yield great results despite all of the state-of-the-art improvements, with over 50% of the tumour sections in the dataset being missed. Our work highlights the discrepancy between the commonly used patch-based analysis and the overall performance on the whole image and the importance of focusing on the metrics relevant to the ultimate user − in our case, the clinician. Much work remains to be done to bring state-of-the-art segmentation to the clinical practice of cancer screening.

[1]  Mohammad Zalbagi Darestani,et al.  MONAI: An open-source framework for deep learning in healthcare , 2022, ArXiv.

[2]  M. Berger,et al.  Preoperative Brain Tumor Imaging: Models and Software for Segmentation and Standardized Reporting , 2022, Frontiers in Neurology.

[3]  Peng Zhang,et al.  A new VAE-GAN model to synthesize arterial spin labeling images from structural MRI , 2021, Displays.

[4]  J. Dowling,et al.  A review of medical image data augmentation techniques for deep learning applications , 2021, Journal of medical imaging and radiation oncology.

[5]  Jonathan Ventura,et al.  Evaluation of MRI Denoising Methods Using Unsupervised Learning , 2021, Frontiers in Artificial Intelligence.

[6]  Daniel Gutierrez-Galan,et al.  Wide & Deep neural network model for patch aggregation in CNN-based prostate cancer detection systems , 2021, Comput. Biol. Medicine.

[7]  Anthonin Reilhac,et al.  Comparison of metrics for the evaluation of medical segmentations using prostate MRI dataset , 2021, Comput. Biol. Medicine.

[8]  Anna Goldenberg,et al.  3D Reasoning for Unsupervised Anomaly Detection in Pediatric WbMRI , 2021, ArXiv.

[9]  Carola-Bibiane Schönlieb,et al.  Unified Focal loss: Generalising Dice and cross entropy-based losses to handle class imbalanced medical image segmentation , 2021, Comput. Medical Imaging Graph..

[10]  Aaron Carass,et al.  Dual-Cycle Constrained Bijective Vae-Gan For Tagged-To-Cine Magnetic Resonance Image Synthesis , 2021, 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI).

[11]  Anne L. Martel,et al.  Overcoming the limitations of patch-based learning to detect cancer in whole slide images , 2020, Scientific Reports.

[12]  Serkan Kiranyaz,et al.  Exploring the effect of image enhancement techniques on COVID-19 detection using chest X-ray images , 2020, Computers in Biology and Medicine.

[13]  Shantala Giraddi,et al.  Breast Cancer Detection Using GAN for Limited Labeled Dataset , 2020, 2020 12th International Conference on Computational Intelligence and Communication Networks (CICN).

[14]  Aditya Khamparia,et al.  DCAVN: Cervical cancer prediction and classification using deep convolutional and variational autoencoder network , 2020, Multimedia Tools and Applications.

[15]  S. Murugan,et al.  Automatic Skin Tumour Segmentation Using Prioritized Patch Based Region – A Novel Comparative Technique , 2020, IETE Journal of Research.

[16]  Jitendra Jonnagaddala,et al.  Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks , 2020, Medical Image Anal..

[17]  H. P. Kok,et al.  Deep learning‐based reconstruction of in vivo pelvis conductivity with a 3D patch‐based convolutional neural network trained on simulated MR data , 2020, Magnetic resonance in medicine.

[18]  Lipo Wang,et al.  3D Deep Learning on Medical Images: A Review , 2020, Sensors.

[19]  Ahmet Çinar,et al.  Detection of tumors on brain MRI images using the hybrid convolutional neural network architecture. , 2020, Medical hypotheses.

[20]  Brojo Kishore Mishra,et al.  Brain Tumor Detection and Classification Using Convolutional Neural Network and Deep Neural Network , 2020, 2020 International Conference on Computer Science, Engineering and Applications (ICCSEA).

[21]  Thomas J. Fuchs,et al.  Beyond Classification: Whole Slide Tissue Histopathology Analysis By End-To-End Part Learning , 2020, MIDL.

[22]  Anna Goldenberg,et al.  Using Generative Models for Pediatric wbMRI , 2020, ArXiv.

[23]  Zhenan Sun,et al.  A3GAN: An Attribute-Aware Attentive Generative Adversarial Network for Face Aging , 2019, IEEE Transactions on Information Forensics and Security.

[24]  Rasoul Sadeghian,et al.  Generating Synthetic Medical Images by Using GAN to Improve CNN Performance in Skin Cancer Classification , 2019, 2019 7th International Conference on Robotics and Mechatronics (ICRoM).

[25]  Kento Aida,et al.  Gastric Cancer Detection from Endoscopic Images Using Synthesis by GAN , 2019, MICCAI.

[26]  Tonmoy Hossain,et al.  Brain Tumor Detection Using Convolutional Neural Network , 2019, 2019 1st International Conference on Advances in Science, Engineering and Robotics Technology (ICASERT).

[27]  Paul Babyn,et al.  Generative Adversarial Network in Medical Imaging: A Review , 2018, Medical Image Anal..

[28]  Hussna Elnoor Mohammed Abdalla,et al.  Brain Tumor Detection by using Artificial Neural Network , 2018, 2018 International Conference on Computer, Control, Electrical, and Electronics Engineering (ICCCEEE).

[29]  Tao Zhang,et al.  Fully automatic and robust segmentation of the clinical target volume for radiotherapy of breast cancer using big data and deep learning. , 2018, Physica medica : PM : an international journal devoted to the applications of physics to medicine and biology : official journal of the Italian Association of Biomedical Physics.

[30]  David M. Thomas,et al.  Baseline Surveillance in Li-Fraumeni Syndrome Using Whole-Body Magnetic Resonance Imaging: A Meta-analysis , 2017, JAMA oncology.

[31]  H. Aerts,et al.  Deep Learning for Fully-Automated Localization and Segmentation of Rectal Cancer on Multiparametric MR , 2017, Scientific Reports.

[32]  Junzhou Huang,et al.  Deep convolutional neural network for survival analysis with pathological images , 2016, 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[33]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[34]  Allan Hanbury,et al.  Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool , 2015, BMC Medical Imaging.

[35]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[36]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[37]  Ronald L. Graham,et al.  An Efficient Algorithm for Determining the Convex Hull of a Finite Planar Set , 1972, Inf. Process. Lett..

[38]  Kerstin Ritter,et al.  MRI Image Registration Considerably Improves CNN-Based Disease Classification , 2021, MLCN@MICCAI.

[39]  Fahed Abdallah,et al.  A Surprisingly Effective Perimeter-based Loss for Medical Image Segmentation , 2021, MIDL.

[40]  Juanjuan Zhao,et al.  Tumour growth prediction of follow-up lung cancer via conditional recurrent variational autoencoder , 2020, IET Image Process..

[41]  Alex Chang,et al.  Detecting Early Stage Lung Cancer using a Neural Network Trained with Patches from Synthetically Generated X-Rays , 2019 .

[42]  A. Moturu,et al.  Creation of Synthetic X-Rays to Train a Neural Network to Detect Lung Cancer , 2018 .