Data Augmentation for Brain-Tumor Segmentation: A Review

Data augmentation is a popular technique which helps improve generalization capabilities of deep neural networks, and can be perceived as implicit regularization. It plays a pivotal role in scenarios in which the amount of high-quality ground-truth data is limited, and acquiring new examples is costly and time-consuming. This is a very common problem in medical image analysis, especially tumor delineation. In this paper, we review the current advances in data-augmentation techniques applied to magnetic resonance images of brain tumors. To better understand the practical aspects of such algorithms, we investigate the papers submitted to the Multimodal Brain Tumor Segmentation Challenge (BraTS 2018 edition), as the BraTS dataset became a standard benchmark for validating existent and emerging brain-tumor detection and segmentation techniques. We verify which data augmentation approaches were exploited and what was their impact on the abilities of underlying supervised learners. Finally, we highlight the most promising research directions to follow in order to synthesize high-quality artificial brain-tumor examples which can boost the generalization abilities of deep models.

[1]  Kurt Keutzer,et al.  A Novel Domain Adaptation Framework for Medical Image Segmentation , 2018, BrainLes@MICCAI.

[2]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[3]  Ole Solheim,et al.  Glioblastoma Segmentation: Comparison of Three Different Software Packages , 2016, PloS one.

[4]  Lei Wang,et al.  3D cGAN based cross-modality MR image synthesis for brain tumor segmentation , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[5]  Nikos Paragios,et al.  Context Aware 3D CNNs for Brain Tumor Segmentation , 2018, BrainLes@MICCAI.

[6]  Chang Liu,et al.  Automatic Brain Tumor Segmentation and Overall Survival Prediction Using Machine Learning Algorithms , 2018, BrainLes@MICCAI.

[7]  Joon Hyuk Cha,et al.  Deep Learning-Based Deep Brain Stimulation Targeting and Clinical Applications , 2019, Front. Neurosci..

[8]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[9]  Hayit Greenspan,et al.  GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification , 2018, Neurocomputing.

[10]  Jakub Nalepa,et al.  Fully-automated deep learning-powered system for DCE-MRI analysis of brain tumors , 2019, Artif. Intell. Medicine.

[11]  Rashima Mahajan,et al.  Medical Images Contrast Enhancement using Quad Weighted Histogram Equalization with Adaptive Gama Correction and Homomorphic Filtering , 2017 .

[12]  Taghi M. Khoshgoftaar,et al.  A survey on Image Data Augmentation for Deep Learning , 2019, Journal of Big Data.

[13]  Jakub Nalepa,et al.  Segmenting Brain Tumors from MRI Using Cascaded Multi-modal U-Nets , 2018, BrainLes@MICCAI.

[14]  G. G. Lakshmi Priya,et al.  Automated brain tumour segmentation techniques— A review , 2017, Int. J. Imaging Syst. Technol..

[15]  Fei-Yue Wang,et al.  Generative adversarial networks: introduction and outlook , 2017, IEEE/CAA Journal of Automatica Sinica.

[16]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[17]  Spyridon Bakas,et al.  Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries , 2018, Lecture Notes in Computer Science.

[18]  Joseph Chazalon,et al.  Segmentation of Gliomas and Prediction of Patient Overall Survival: A Simple and Fast Procedure , 2018, BrainLes@MICCAI.

[19]  Jun Ma,et al.  Automatic Brain Tumor Segmentation by Exploring the Multi-modality Complementary Information and Cascaded 3D Lightweight CNNs , 2018, BrainLes@MICCAI.

[20]  Jaime S. Cardoso,et al.  Elastic deformations for data augmentation in breast cancer mass detection , 2018, 2018 IEEE EMBS International Conference on Biomedical & Health Informatics (BHI).

[21]  Klaus H. Maier-Hein,et al.  No New-Net , 2018, 1809.10483.

[22]  Ender Konukoglu,et al.  Semi-Supervised and Task-Driven Data Augmentation , 2019, IPMI.

[23]  Tarek Khadir,et al.  Deep Convolutional Neural Networks Using U-Net for Automatic Brain Tumor Segmentation in Multimodal MRI Volumes , 2018, BrainLes@MICCAI.

[24]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[25]  Ling Liu,et al.  Adversarial Examples in Deep Learning: Characterization and Divergence , 2018, ArXiv.

[26]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[27]  Shin'ichi Satoh,et al.  Learning More with Less: Conditional PGGAN-based Data Augmentation for Brain Metastases Detection Using Highly-Rough Annotation on MR Images , 2019, CIKM.

[28]  Hongtu Zhu,et al.  Automatic Brain Tumor Segmentation with Domain Adaptation , 2018, BrainLes@MICCAI.

[29]  M. Jorge Cardoso,et al.  Improving Data Augmentation for Medical Image Segmentation , 2018 .

[30]  V. E. Turlapov,et al.  Glioma Segmentation with Cascaded Unet , 2018, BrainLes@MICCAI.

[31]  Christos Davatzikos,et al.  Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features , 2017, Scientific Data.

[32]  Guy Amit,et al.  Classification of breast MRI lesions using small-size training sets: comparison of deep learning approaches , 2017, Medical Imaging.

[33]  Raghav Mehta,et al.  3D U-Net for Brain Tumour Segmentation , 2018, BrainLes@MICCAI.

[34]  Li Sun,et al.  Tumor Segmentation and Survival Prediction in Glioma with Deep Learning , 2018, BrainLes@MICCAI.

[35]  Brian B. Avants,et al.  The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) , 2015, IEEE Transactions on Medical Imaging.

[36]  Sébastien Ourselin,et al.  Automatic Brain Tumor Segmentation using Convolutional Neural Networks with Test-Time Augmentation , 2018, BrainLes@MICCAI.

[37]  Terrance E. Boult,et al.  Towards Robust Deep Neural Networks with BANG , 2016, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[38]  Mark D. McDonnell,et al.  Understanding Data Augmentation for Classification: When to Warp? , 2016, 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA).

[39]  Michael Miller,et al.  Unbiased Diffeomorphic Mapping of Longitudinal Data with Simultaneous Subject Specific Template Estimation , 2017, GRAIL/MFCA/MICGen@MICCAI.

[40]  F. Barkhof,et al.  Inter-rater agreement in glioma segmentations on longitudinal MRI , 2019, NeuroImage: Clinical.

[41]  Ahmed Ben Hamida,et al.  A comparative study of MRI contrast enhancement techniques based on Traditional Gamma Correction and Adaptive Gamma Correction: Case of multiple sclerosis pathology , 2018, 2018 4th International Conference on Advanced Technologies for Signal and Image Processing (ATSIP).

[42]  Tony P. Pridmore,et al.  Deep Hourglass for Brain Tumor Segmentation , 2018, BrainLes@MICCAI.

[43]  Andriy Myronenko,et al.  3D MRI brain tumor segmentation using autoencoder regularization , 2018, BrainLes@MICCAI.

[44]  William Hollingworth,et al.  Interrater reliability in assessing quality of diagnostic accuracy studies using the QUADAS tool. A preliminary assessment. , 2006, Academic radiology.

[45]  B. S. Manjunath,et al.  Brain Tumor Segmentation and Tractographic Feature Extraction from Structural MR Images for Overall Survival Prediction , 2018, BrainLes@MICCAI.

[46]  Pedro Costa,et al.  Data-Driven Color Augmentation Techniques for Deep Skin Image Analysis , 2017, ArXiv.

[47]  Cem Direkoglu,et al.  Review of MRI-based Brain Tumor Image Segmentation Using Deep Learning Methods , 2016 .

[48]  Poonam Bhogale,et al.  Brain Tumor Segmentation , 2016 .

[49]  et al.,et al.  Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge , 2018, ArXiv.

[50]  Changming Sun,et al.  Supervised Brain Tumor Segmentation Based on Gradient and Context-Sensitive Features , 2019, Front. Neurosci..

[51]  Brian B. Avants,et al.  Directly Manipulated Free-Form Deformation Image Registration , 2009, IEEE Transactions on Image Processing.

[52]  Shin'ichi Satoh,et al.  Learning More with Less: GAN-based Medical Image Augmentation , 2019, ArXiv.

[53]  Victor Alves,et al.  Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images , 2016, IEEE Transactions on Medical Imaging.

[54]  Jeffrey L. Gunter,et al.  Medical Image Synthesis for Data Augmentation and Anonymization using Generative Adversarial Networks , 2018, SASHIMI@MICCAI.

[55]  Sébastien Ourselin,et al.  Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks , 2018, Neurocomputing.

[56]  Mohammed Bennamoun,et al.  A Multi-Modal, Discriminative and Spatially Invariant CNN for RGB-D Object Labeling , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[57]  Jakub Nalepa,et al.  Data Augmentation via Image Registration , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[58]  Hongyi Zhang,et al.  mixup: Beyond Empirical Risk Minimization , 2017, ICLR.

[59]  Parashkev Nachev,et al.  Computer Methods and Programs in Biomedicine NiftyNet: a deep-learning platform for medical imaging , 2022 .

[60]  Sabine Van Huffel,et al.  Semi-automated brain tumor segmentation on multi-parametric MRI using regularized non-negative matrix factorization , 2017, BMC Medical Imaging.

[61]  Albert C. S. Chung,et al.  Learning Data Augmentation for Brain Tumor Segmentation with Coarse-to-Fine Generative Adversarial Networks , 2018, BrainLes@MICCAI.

[62]  Sachin Mehta,et al.  3D-ESPNet with Pyramidal Refinement for Volumetric Brain Tumor Image Segmentation , 2018, BrainLes@MICCAI.

[63]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[64]  Daniel L. Rubin,et al.  Differential Data Augmentation Techniques for Medical Imaging Classification Tasks , 2017, AMIA.

[65]  Christoph Meinel,et al.  Conditional Adversarial Network for Semantic Segmentation of Brain Tumor , 2017, ArXiv.

[66]  Jakub Nalepa,et al.  Segmenting brain tumors from FLAIR MRI using fully convolutional neural networks , 2019, Comput. Methods Programs Biomed..

[67]  René M. Botnar,et al.  Automatic CNN-based detection of cardiac MR motion artefacts using k-space data augmentation and curriculum learning , 2018, Medical Image Anal..

[68]  Cordelia Schmid,et al.  On the Importance of Visual Context for Data Augmentation in Scene Understanding , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[69]  Albert Montillo,et al.  Anatomically-Informed Data Augmentation for functional MRI with Applications to Deep Learning , 2019, Medical Imaging: Image Processing.

[70]  Zhaohui Huang,et al.  Affine-invariant B-spline moments for curve matching , 1996, IEEE Trans. Image Process..

[71]  Makina Yabashi,et al.  Two-colour serial femtosecond crystallography dataset from gadoteridol-derivatized lysozyme for MAD phasing , 2017, Scientific Data.

[72]  Ganapathy Krishnamurthi,et al.  Generative adversarial networks for brain lesion detection , 2017, Medical Imaging.

[73]  Brian B. Avants,et al.  Explicit B-spline regularization in diffeomorphic image registration , 2013, Front. Neuroinform..

[74]  Jakub Nalepa,et al.  Training- and Test-Time Data Augmentation for Hyperspectral Image Segmentation , 2020, IEEE Geoscience and Remote Sensing Letters.

[75]  Steve B. Jiang,et al.  A deep convolutional neural network-based automatic delineation strategy for multiple brain metastases stereotactic radiosurgery , 2017, PloS one.

[76]  Richard McKinley,et al.  Ensembles of Densely-Connected CNNs with Label-Uncertainty for Brain Tumor Segmentation , 2018, BrainLes@MICCAI.

[77]  Antonio Albiol,et al.  Extending 2D Deep Learning Architectures to 3D Image Segmentation Problems , 2018, BrainLes@MICCAI.

[78]  Christoph Meinel,et al.  voxel-GAN: Adversarial Framework for Learning Imbalanced Brain Tumor Segmentation , 2018, BrainLes@MICCAI.

[79]  David Gur,et al.  Bidirectional elastic image registration using B-spline affine transformation , 2014, Comput. Medical Imaging Graph..

[80]  Nicholas J. Tustison,et al.  Brain Tumor Segmentation Using an Ensemble of 3D U-Nets and Overall Survival Prediction Using Radiomic Features , 2018, Frontiers in Computational Neuroscience.