AbdomenCT-1K: Is Abdominal Organ Segmentation A Solved Problem?

With the unprecedented developments in deep learning, automatic segmentation of main abdominal organs (i.e., liver, kidney, and spleen) seems to be a solved problem as the state-of-the-art (SOTA) methods have achieved comparable results with inter-observer variability on existing benchmark datasets. However, most of the existing abdominal organ segmentation benchmark datasets only contain single-center, single-phase, single-vendor, or single-disease cases, thus, it is unclear whether the excellent performance can generalize on more diverse datasets. In this paper, we present a large and diverse abdominal CT organ segmentation dataset, termed as AbdomenCT-1K, with more than 1000 (1K) CT scans from 11 countries, including multi-center, multi-phase, multi-vendor, and multi-disease cases. Furthermore, we conduct a large-scale study for liver, kidney, spleen, and pancreas segmentation, as well as reveal the unsolved segmentation problems of the SOTA method, such as the limited generalization ability on distinct medical centers, phases, and unseen diseases. To advance the unsolved problems, we build four organ segmentation benchmarks for fully supervised, semi-supervised, weakly supervised, and continual learning, which are currently challenging and active research topics. Accordingly, we develop a simple and effective method for each benchmark, which can be used as out-of-the-box methods and strong baselines. We believe the introduction of the AbdomenCT-1K dataset will promote future in-depth research towards clinical applicable abdominal organ segmentation methods. Moreover, the datasets, codes, and trained models of baseline methods will be publicly available at this https URL.

[1]  Bennett A. Landman,et al.  Semi-Supervised Multi-Organ Segmentation through Quality Assurance Supervision , 2019, Medical Imaging: Image Processing.

[2]  Mert R. Sabuncu,et al.  Multi-atlas segmentation of biomedical images: A survey , 2014, Medical Image Anal..

[3]  Mrityunjaya V. Latte,et al.  Combined endeavor of Neutrosophic Set and Chan-Vese model to extract accurate liver image from CT scan , 2017, Comput. Methods Programs Biomed..

[4]  Hans-Peter Meinzer,et al.  Computerized planning of liver surgery - an overview , 2002, Comput. Graph..

[5]  Davide Maltoni,et al.  CORe50: a New Dataset and Benchmark for Continuous Object Recognition , 2017, CoRL.

[6]  Yan Shen,et al.  Scribble-Based Hierarchical Weakly Supervised Learning for Brain Tumor Segmentation , 2019, MICCAI.

[7]  Dong-Hyun Lee,et al.  Pseudo-Label : The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks , 2013 .

[8]  Quoc V. Le,et al.  Self-Training With Noisy Student Improves ImageNet Classification , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Xinjian Chen,et al.  Automatic Liver Segmentation Based on Shape Constraints and Deformable Graph Cut in CT Images , 2015, IEEE Transactions on Image Processing.

[10]  Dean C. Barratt,et al.  Automatic Multi-Organ Segmentation on Abdominal CT With Dense V-Networks , 2018, IEEE Transactions on Medical Imaging.

[11]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[12]  Liang Zhang,et al.  Block Level Skip Connections Across Cascaded V-Net for Multi-Organ Segmentation , 2020, IEEE Transactions on Medical Imaging.

[13]  Benedikt Pfülb,et al.  A comprehensive, application-oriented study of catastrophic forgetting in DNNs , 2019, ICLR.

[14]  et al.,et al.  Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge , 2018, ArXiv.

[15]  Fei-Fei Li,et al.  What's the Point: Semantic Segmentation with Point Supervision , 2015, ECCV.

[16]  Trevor Darrell,et al.  Constrained Convolutional Neural Networks for Weakly Supervised Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[17]  Holger H. Hoos,et al.  A survey on semi-supervised learning , 2019, Machine Learning.

[18]  Josien P. W. Pluim,et al.  Not‐so‐supervised: A survey of semi‐supervised, multi‐instance, and transfer learning in medical image analysis , 2018, Medical Image Anal..

[19]  Xinlei Chen,et al.  Prior-Aware Neural Network for Partially-Supervised Multi-Organ Segmentation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[20]  Luc Van Gool,et al.  A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Marius George Linguraru,et al.  Abdominal multi-organ segmentation from CT images using conditional shape-location and unsupervised intensity priors , 2015, Medical Image Anal..

[22]  Xu Jia,et al.  Continual learning: A comparative study on how to defy forgetting in classification tasks , 2019, ArXiv.

[23]  Seong-Whan Lee,et al.  Cascaded MultiTask 3-D Fully Convolutional Networks for Pancreas Segmentation , 2019, IEEE Transactions on Cybernetics.

[24]  Bram van Ginneken,et al.  Fully Automatic Volume Measurement of the Spleen at CT Using Deep Learning. , 2020, Radiology. Artificial intelligence.

[25]  Yunchao Wei,et al.  Weakly Supervised Scene Parsing with Point-based Distance Metric Learning , 2018, AAAI.

[26]  Xiangrong Zhou,et al.  Construction of a probabilistic atlas for automated liver segmentation in non-contrast torso CT images , 2005 .

[27]  Ashish Verma,et al.  An Automated System for Atlas Based Multiple OrganSegmentation of Abdominal CT Images , 2016 .

[28]  L. Joskowicz,et al.  Inter-observer variability of manual contour delineation of structures in CT , 2018, European Radiology.

[29]  Fredrik Kahl,et al.  Robust Abdominal Organ Segmentation Using Regional Convolutional Neural Networks , 2017, SCIA.

[30]  Bennett A Landman,et al.  Efficient multi-atlas abdominal segmentation on clinically acquired CT with SIMPLE context learning , 2015, Medical Image Anal..

[31]  Jonathon Shlens,et al.  Leveraging Semi-Supervised Learning in Video Sequences for Urban Scene Segmentation , 2020, ECCV.

[32]  Luc Van Gool,et al.  The 2019 DAVIS Challenge on VOS: Unsupervised Multi-Object Segmentation , 2019, ArXiv.

[33]  Yan Wang,et al.  A Fixed-Point Model for Pancreas Segmentation in Abdominal CT Scans , 2016, MICCAI.

[34]  Nima Tajbakhsh,et al.  Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation , 2019, Medical Image Anal..

[35]  Jian Sun,et al.  ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  J. Mongan,et al.  Checklist for Artificial Intelligence in Medical Imaging (CLAIM): A Guide for Authors and Reviewers. , 2020, Radiology. Artificial intelligence.

[37]  Huimin Ma,et al.  Weakly-Supervised Semantic Segmentation by Iterative Affinity Learning , 2020, International Journal of Computer Vision.

[38]  Daniel Rueckert,et al.  Discriminative dictionary learning for abdominal multi-organ segmentation , 2015, Medical Image Anal..

[39]  Yuichiro Hayashi,et al.  An application of cascaded 3D fully convolutional networks for medical image segmentation , 2018, Comput. Medical Imaging Graph..

[40]  Hao Chen,et al.  The Liver Tumor Segmentation Benchmark (LiTS) , 2019, Medical Image Anal..

[41]  Stefan Wermter,et al.  Continual Lifelong Learning with Neural Networks: A Review , 2019, Neural Networks.

[42]  Martin Styner,et al.  Comparison and Evaluation of Methods for Liver Segmentation From CT Datasets , 2009, IEEE Transactions on Medical Imaging.

[43]  Yang Wen,et al.  Weakly Supervised Learning of Recurrent Residual ConvNets for Pancreas Segmentation in CT Scans , 2019, 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[44]  Andreas Nürnberger,et al.  CHAOS Challenge - Combined (CT-MR) Healthy Abdominal Organ Segmentation , 2020, Medical Image Anal..

[45]  S. Kevin Zhou,et al.  Marginal loss and exclusion loss for partially supervised multi-organ segmentation , 2020, Medical Image Anal..

[46]  Geraint Rees,et al.  Deep learning to achieve clinically applicable segmentation of head and neck anatomy for radiotherapy , 2018, ArXiv.

[47]  Hai Huang,et al.  Precision Liver Resection: Three-Dimensional Reconstruction Combined with Fluorescence Laparoscopic Imaging , 2020, Surgical innovation.

[48]  Xing Zhang,et al.  Automatic Liver Segmentation Using a Statistical Shape Model With Optimal Surface Detection , 2010, IEEE Transactions on Biomedical Engineering.

[49]  Giorgio Metta,et al.  Incremental robot learning of new objects with fixed update time , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[50]  Yannis Avrithis,et al.  Label Propagation for Deep Semi-Supervised Learning , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Yan Wang,et al.  Abdominal multi-organ segmentation with organ-attention networks and statistical fusion , 2018, Medical Image Anal..

[52]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[53]  Klaus H. Maier-Hein,et al.  Automated Design of Deep Learning Methods for Biomedical Image Segmentation , 2019 .

[54]  B. van Ginneken,et al.  Computer-aided diagnosis: how to move from the laboratory to the clinic. , 2011, Radiology.

[55]  Yan Huang,et al.  Box-Driven Class-Wise Region Masking and Filling Rate Guided Loss for Weakly Supervised Semantic Segmentation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[56]  Feng Chen,et al.  Automatic abdominal multi-organ segmentation using deep convolutional neural network and time-implicit level sets , 2016, International Journal of Computer Assisted Radiology and Surgery.

[57]  Suchi Saria,et al.  Minimum information about clinical artificial intelligence modeling: the MI-CLAIM checklist , 2020, Nature Medicine.

[58]  Ronald M. Summers,et al.  A Review of Deep Learning in Medical Imaging: Imaging Traits, Technology Trends, Case Studies With Progress Highlights, and Future Promises , 2020, Proceedings of the IEEE.

[59]  Michael McCloskey,et al.  Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem , 1989 .

[60]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[61]  Lei Xing,et al.  Modified U-Net (mU-Net) With Incorporation of Object-Dependent High Level Features for Improved Liver and Liver-Tumor Segmentation in CT Images , 2019, IEEE Transactions on Medical Imaging.

[62]  Ronald M. Summers,et al.  A large annotated medical image dataset for the development and evaluation of segmentation algorithms , 2019, ArXiv.

[63]  Yaozong Gao,et al.  The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 Challenge , 2019, Medical Image Anal..

[64]  Nikolaos Papanikolopoulos,et al.  An international challenge to use artificial intelligence to define the state-of-the-art in kidney and kidney tumor segmentation in CT imaging. , 2020 .

[65]  Yuichiro Hayashi,et al.  A multi-scale pyramid of 3D fully convolutional networks for abdominal multi-organ segmentation , 2018, MICCAI.

[66]  Hongbo Zhang,et al.  3D liver segmentation using multiple region appearances and graph cuts. , 2015, Medical physics.

[67]  Song Wang,et al.  Three-Dimensional CT Image Segmentation by Combining 2D Fully Convolutional Network with 3D Majority Voting , 2016, LABELS/DLMIA@MICCAI.

[68]  Nassim Bouteldja,et al.  OBELISK‐Net: Fewer layers to solve 3D multi‐organ segmentation with sparse deformable convolutions , 2019, Medical Image Anal..

[69]  Ben Glocker,et al.  Joint Supervoxel Classification Forest for Weakly-Supervised Organ Segmentation , 2017, MLMI@MICCAI.

[70]  Jonathan Sykes,et al.  Reflections on the current status of commercial automated segmentation systems in clinical practice , 2014, Journal of medical radiation sciences.

[71]  Xiaoping Yang,et al.  Learning Geodesic Active Contours for Embedding Object Global Information in Segmentation CNNs , 2020, IEEE Transactions on Medical Imaging.

[72]  Wei Shen,et al.  Semi-Supervised 3D Abdominal Multi-Organ Segmentation Via Deep Multi-Planar Co-Training , 2018, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV).

[73]  Xin Yang,et al.  Deep Learning Techniques for Automatic MRI Cardiac Multi-Structures Segmentation and Diagnosis: Is the Problem Solved? , 2018, IEEE Transactions on Medical Imaging.

[74]  Yoshua Bengio,et al.  An Empirical Investigation of Catastrophic Forgeting in Gradient-Based Neural Networks , 2013, ICLR.

[75]  Jie Li,et al.  A sparse annotation strategy based on attention-guided active learning for 3D medical image segmentation , 2019, ArXiv.

[76]  Ming-Ming Cheng,et al.  Leveraging Instance-, Image- and Dataset-Level Information for Weakly Supervised Instance Segmentation , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[77]  Luc Van Gool,et al.  The 2017 DAVIS Challenge on Video Object Segmentation , 2017, ArXiv.