Cutting-edge 3D Medical Image Segmentation Methods in 2020: Are Happy Families All Alike?

Segmentation is one of the most important and popular tasks in medical image analysis, which plays a critical role in disease diagnosis, surgical planning, and prognosis evaluation. During the past five years, on the one hand, thousands of medical image segmentation methods have been proposed for various organs and lesions in different medical images, which become more and more challenging to fairly compare different methods. On the other hand, international segmentation challenges can provide a transparent platform to fairly evaluate and compare different methods. In this paper, we present a comprehensive review of the top methods in ten 3D medical image segmentation challenges during 2020, covering a variety of tasks and datasets. We also identify the ”happy-families” practices in the cutting-edge segmentation methods, which are useful for developing powerful segmentation approaches. Finally, we discuss open research problems that should be addressed in the future. We also maintain a list of cutting-edge segmentation methods at https://github.com/JunMa11/SOTA-MedSeg.

[1]  Jun Ma,et al.  Histogram Matching Augmentation for Domain Adaptation with Application to Multi-Centre, Multi-Vendor and Multi-Disease Cardiac Image Segmentation , 2020, M&Ms and EMIDEC/STACOM@MICCAI.

[2]  George T. Y. Chen,et al.  Four-dimensional image-based treatment planning: Target volume segmentation and dose calculation in the presence of respiratory motion. , 2005, International journal of radiation oncology, biology, physics.

[3]  Vincent Andrearczyk,et al.  Automatic Segmentation of Head and Neck Tumors and Nodal Metastases in PET-CT scans , 2020, MIDL.

[4]  Deniz Erdogmus,et al.  Tversky Loss Function for Image Segmentation Using 3D Fully Convolutional Deep Networks , 2017, MLMI@MICCAI.

[5]  Antonio J. Plaza,et al.  Image Segmentation Using Deep Learning: A Survey , 2021, IEEE transactions on pattern analysis and machine intelligence.

[6]  Dong Wang,et al.  A Fast Algorithm for Geodesic Active Contours with Applications to Medical Image Segmentation , 2020, ArXiv.

[7]  Jan S. Kirschke,et al.  VerSe: A Vertebrae Labelling and Segmentation Benchmark , 2020, ArXiv.

[8]  Xiahai Zhuang,et al.  Multivariate Mixture Model for Myocardial Segmentation Combining Multi-Source Images , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Xiahai Zhuang,et al.  Multivariate Mixture Model for Cardiac Segmentation from Multi-Sequence MRI , 2016, MICCAI.

[10]  Dong Wang,et al.  The iterative convolution-thresholding method (ICTM) for image segmentation , 2019, Pattern Recognit..

[11]  Vincent Andrearczyk,et al.  Overview of the HECKTOR Challenge at MICCAI 2020: Automatic Head and Neck Tumor Segmentation in PET/CT , 2020, HECKTOR@MICCAI.

[12]  Holger H. Hoos,et al.  A survey on semi-supervised learning , 2019, Machine Learning.

[13]  Josien P. W. Pluim,et al.  Not‐so‐supervised: A survey of semi‐supervised, multi‐instance, and transfer learning in medical image analysis , 2018, Medical Image Anal..

[14]  Sebastian Risi,et al.  Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks , 2017, Neural Networks.

[15]  Anonymous Name,et al.  How Distance Transform Maps Boost Segmentation CNNs: An Empirical Study , 2020 .

[16]  Haohan Li,et al.  An efficient iterative thresholding method for image segmentation , 2016, J. Comput. Phys..

[17]  et al.,et al.  Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge , 2018, ArXiv.

[18]  Jose Dolz,et al.  Boundary loss for highly unbalanced segmentation , 2018, MIDL.

[19]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[20]  Christopher Joseph Pal,et al.  The Importance of Skip Connections in Biomedical Image Segmentation , 2016, LABELS/DLMIA@MICCAI.

[21]  Dacheng Tao,et al.  Recent advances in deep learning theory , 2020, ArXiv.

[22]  Jun Ma,et al.  Cascaded Framework for Automatic Evaluation of Myocardial Infarction from Delayed-Enhancement Cardiac MRI , 2020, ArXiv.

[23]  Brian B. Avants,et al.  The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) , 2015, IEEE Transactions on Medical Imaging.

[24]  Peter M. Full,et al.  Studying Robustness of Semantic Segmentation under Domain Shift in cardiac MRI , 2020, M&Ms and EMIDEC/STACOM@MICCAI.

[25]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[26]  Klaus H. Maier-Hein,et al.  nnU-Net for Brain Tumor Segmentation , 2020, BrainLes@MICCAI.

[27]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[28]  Cheng Bian,et al.  Ensembled ResUnet for Anatomical Brain Barriers Segmentation , 2020, ArXiv.

[29]  Jun Ma,et al.  Exploring Large Context for Cerebral Aneurysm Segmentation , 2020, ArXiv.

[30]  Yan Wang,et al.  Deep Distance Transform for Tubular Structure Segmentation in CT Scans , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Christian Payer,et al.  Integrating spatial configuration into heatmap regression based CNNs for landmark localization , 2019, Medical Image Anal..

[32]  Christos Davatzikos,et al.  Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features , 2017, Scientific Data.

[33]  Lena Maier-Hein,et al.  BIAS: Transparent reporting of biomedical image analysis challenges , 2019, Medical Image Analysis.

[34]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[35]  Jiebo Luo,et al.  Small Data Challenges in Big Data Era: A Survey of Recent Progress on Unsupervised and Semi-Supervised Methods , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Guotai Wang,et al.  Myocardial Edema and Scar Segmentation Using a Coarse-to-Fine Framework with Weighted Ensemble , 2020, MyoPS@MICCAI.

[37]  Stefan Wermter,et al.  Continual Lifelong Learning with Neural Networks: A Review , 2019, Neural Networks.

[38]  Lei Wu,et al.  Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don't , 2020, CSIAM Transactions on Applied Mathematics.

[39]  Guang-Zhong Yang,et al.  ACNN: a Full Resolution DCNN for Medical Image Segmentation , 2019, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[40]  Paul Kinahan,et al.  Radiomics: Images Are More than Pictures, They Are Data , 2015, Radiology.

[41]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Nima Tajbakhsh,et al.  Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation , 2019, Medical Image Anal..

[43]  Alain Lalande,et al.  Emidec: A Database Usable for the Automatic Evaluation of Myocardial Infarction from Delayed-Enhancement Cardiac MRI , 2020, Data.

[44]  W. Liang,et al.  Clinically Applicable AI System for Accurate Diagnosis, Quantitative Measurements, and Prognosis of COVID-19 Pneumonia Using Computed Tomography , 2020, Cell.

[45]  Xiaoping Yang,et al.  Combining CNN and Hybrid Active Contours for Head and Neck Tumor Segmentation in CT and PET images , 2020, HECKTOR@MICCAI.

[46]  Zhiqiang He,et al.  Towards Data-Efficient Learning: A Benchmark for COVID-19 CT Lung and Infection Segmentation. , 2020, Medical physics.

[47]  X. Li,et al.  AGE Challenge: Angle Closure Glaucoma Evaluation in Anterior Segment Optical Coherence Tomography , 2020, Medical Image Anal..

[48]  Jens Petersen,et al.  nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation , 2020, Nature Methods.

[49]  Zhiqiang He,et al.  Semi-supervised Cardiac Image Segmentation via Label Propagation and Style Transfer , 2020, ArXiv.

[50]  Weidong Cai,et al.  H2NF-Net for Brain Tumor Segmentation using Multimodal MR Imaging: 2nd Place Solution to BraTS Challenge 2020 Segmentation Task , 2020, BrainLes@MICCAI.

[51]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[52]  Zhiqiang He,et al.  Modality-Pairing Learning for Brain Tumor Segmentation , 2020, ArXiv.

[53]  Qianli Liao,et al.  Theoretical issues in deep networks , 2020, Proceedings of the National Academy of Sciences.

[54]  Steven C. H. Hoi,et al.  Online Learning: A Comprehensive Survey , 2018, Neurocomputing.

[55]  Jian Yang,et al.  Selective Kernel Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[56]  Congcong Wang,et al.  AbdomenCT-1K: Is Abdominal Organ Segmentation a Solved Problem? , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[57]  Hao Chen,et al.  A Multi-Organ Nucleus Segmentation Challenge , 2020, IEEE Transactions on Medical Imaging.

[58]  Jun Ma,et al.  Segmentation Loss Odyssey , 2020, ArXiv.

[59]  Klaus H. Maier-Hein,et al.  Comparative validation of multi-instance instrument segmentation in endoscopy: Results of the ROBUST-MIS 2019 challenge , 2020, Medical Image Anal..

[60]  Yichi Zhang,et al.  Cascaded Convolutional Neural Network for Automatic Myocardial Infarction Segmentation from Delayed-Enhancement Cardiac MRI , 2020, M&Ms and EMIDEC/STACOM@MICCAI.

[61]  Jan S. Kirschke,et al.  Labeling Vertebrae with Two-dimensional Reformations of Multidetector CT Images: An Adversarial Approach for Incorporating Prior Knowledge of Spine Anatomy. , 2020, Radiology. Artificial intelligence.

[62]  Thomas Baum,et al.  A Vertebral Segmentation Dataset with Fracture Grading , 2020, Radiology. Artificial intelligence.

[63]  Simon K. Warfield,et al.  Deep learning with noisy labels: exploring techniques and remedies in medical image analysis , 2020, Medical Image Anal..