Learning Geodesic Active Contours for Embedding Object Global Information in Segmentation CNNs

Most existing CNNs-based segmentation methods rely on local appearances learned on the regular image grid, without consideration of the object global information. This article aims to embed the object global geometric information into a learning framework via the classical geodesic active contours (GAC). We propose a level set function (LSF) regression network, supervised by the segmentation ground truth, LSF ground truth and geodesic active contours, to not only generate the segmentation probabilistic map but also directly minimize the GAC energy functional in an end-to-end manner. With the help of geodesic active contours, the segmentation contour, embedded in the level set function, can be globally driven towards the image boundary to obtain lower energy, and the geodesic constraint can lead the segmentation result to have fewer outliers. Extensive experiments on four public datasets show that (1) compared with state-of-the-art (SOTA) learning active contour methods, our method can achieve significantly better performance; (2) compared with recent SOTA methods that are designed for reducing boundary errors, our method also outperforms them with more accurate boundaries; (3) compared with SOTA methods on two popular multi-class segmentation challenge datasets, our method can still obtain superior or competitive results in both organ and tumor segmentation tasks. Our study demonstrates that introducing global information by GAC can significantly improve segmentation performance, especially on reducing the boundary errors and outliers, which is very useful in applications such as organ transplantation surgical planning and multi-modality image registration where boundary errors can be very harmful.

[1]  Antonio J. Plaza,et al.  Image Segmentation Using Deep Learning: A Survey , 2021, IEEE transactions on pattern analysis and machine intelligence.

[2]  Bryan M. Williams,et al.  Learning Active Contour Models for Medical Image Segmentation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Septimiu E. Salcudean,et al.  Reducing the Hausdorff Distance in Medical Image Segmentation With Convolutional Neural Networks , 2019, IEEE Transactions on Medical Imaging.

[4]  Zongwei Zhou,et al.  Models Genesis , 2020, Medical Image Anal..

[5]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[6]  Chang-Tsun Li,et al.  Weighted Level Set Evolution Based on Local Edge Features for Medical Image Segmentation , 2017, IEEE Transactions on Image Processing.

[7]  Lior Wolf,et al.  Unsupervised Microvascular Image Segmentation Using an Active Contours Mimicking Neural Network , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[8]  Marios Savvides,et al.  Reformulating Level Sets as Deep Recurrent Neural Network Approach to Semantic Segmentation , 2017, IEEE Transactions on Image Processing.

[9]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[11]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Gustavo Carneiro,et al.  Combining deep learning and level set for the automated segmentation of the left ventricle of the heart from cardiac cine magnetic resonance , 2017, Medical Image Anal..

[13]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[14]  Matthew B. Blaschko,et al.  Optimization for Medical Image Segmentation: Theory and Practice When Evaluating With Dice Score or Jaccard Index , 2020, IEEE Transactions on Medical Imaging.

[15]  Ronald M. Summers,et al.  A large annotated medical image dataset for the development and evaluation of segmentation algorithms , 2019, ArXiv.

[16]  Seunghyeon Kim,et al.  CNN-Based Semantic Segmentation Using Level Set Loss , 2019, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV).

[17]  Feng Chen,et al.  Automatic abdominal multi-organ segmentation using deep convolutional neural network and time-implicit level sets , 2016, International Journal of Computer Assisted Radiology and Surgery.

[18]  Guillermo Sapiro,et al.  Geodesic Active Contours , 1995, International Journal of Computer Vision.

[19]  Nima Tajbakhsh,et al.  Models Genesis: Generic Autodidactic Models for 3D Medical Image Analysis , 2019, MICCAI.

[20]  Klaus H. Maier-Hein,et al.  Automated Design of Deep Learning Methods for Biomedical Image Segmentation , 2019 .

[21]  Nuno Vasconcelos,et al.  Volumetric Attention for 3D Medical Image Segmentation and Detection , 2019, MICCAI.

[22]  Richard McKinley,et al.  Ensembles of Densely-Connected CNNs with Label-Uncertainty for Brain Tumor Segmentation , 2018, BrainLes@MICCAI.

[23]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[24]  Jong Chul Ye,et al.  Mumford–Shah Loss Functional for Image Segmentation With Deep Learning , 2019, IEEE Transactions on Image Processing.

[25]  Guido Gerig,et al.  User-guided 3D active contour segmentation of anatomical structures: Significantly improved efficiency and reliability , 2006, NeuroImage.

[26]  Chi-Wing Fu,et al.  H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes , 2018, IEEE Transactions on Medical Imaging.

[27]  Yong Yin,et al.  Shape-Aware Organ Segmentation by Predicting Signed Distance Maps , 2019, AAAI.

[28]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[29]  Lisheng Wang,et al.  Harnessing 2D Networks and 3D Features for Automated Pancreas Segmentation from Volumetric CT Images , 2019, MICCAI.

[30]  Chunming Li,et al.  Medical image segmentation based on level set and isoperimetric constraint. , 2017, Physica medica : PM : an international journal devoted to the applications of physics to medicine and biology : official journal of the Italian Association of Biomedical Physics.

[31]  Hao Chen,et al.  The Liver Tumor Segmentation Benchmark (LiTS) , 2019, Medical Image Anal..

[32]  Xin Yang,et al.  Deep Learning Techniques for Automatic MRI Cardiac Multi-Structures Segmentation and Diagnosis: Is the Problem Solved? , 2018, IEEE Transactions on Medical Imaging.

[33]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[34]  Hao Chen,et al.  Light-Weight Hybrid Convolutional Network for Liver Tumor Segmentation , 2019, IJCAI.

[35]  Chunming Li,et al.  A modified level set algorithm based on point distance shape constraint for lesion and organ segmentation. , 2019, Physica medica : PM : an international journal devoted to the applications of physics to medicine and biology : official journal of the Italian Association of Biomedical Physics.

[36]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[37]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[38]  Chunming Li,et al.  Distance Regularized Level Set Evolution and Its Application to Image Segmentation , 2010, IEEE Transactions on Image Processing.

[39]  Ben Glocker,et al.  Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images , 2018, Medical Image Anal..

[40]  Mohammad Shoeybi,et al.  Neural ODEs for Image Segmentation with Level Sets , 2019, ArXiv.

[41]  Gang Li,et al.  Benchmark on Automatic Six-Month-Old Infant Brain Segmentation Algorithms: The iSeg-2017 Challenge , 2019, IEEE Transactions on Medical Imaging.

[42]  et al.,et al.  Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge , 2018, ArXiv.

[43]  Jose Dolz,et al.  Boundary loss for highly unbalanced segmentation , 2018, MIDL.

[44]  Nima Tajbakhsh,et al.  UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation , 2020, IEEE Transactions on Medical Imaging.

[45]  Richard A. Newcombe,et al.  DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).