Learning 3D Features with 2D CNNs via Surface Projection for CT Volume Segmentation

3D features are desired in nature for segmenting CT volumes. It is, however, computationally expensive to employ a 3D convolutional neural network (CNN) to learn 3D features. Existing methods hence learn 3D features by still relying on 2D CNNs while attempting to consider more 2D slices, but up until now it is difficulty for them to consider the whole volumetric data, resulting in information loss and performance degradation. In this paper, we propose a simple and effective technique that allows a 2D CNN to learn 3D features for segmenting CT volumes. Our key insight is that all boundary voxels of a 3D object form a surface that can be represented by using a 2D matrix, and therefore they can be perfectly recognized by a 2D CNN in theory. We hence learn 3D features for recognizing these boundary voxels by learning the projection distance between a set of prescribed spherical surfaces and the object’s surface, which can be readily performed by a 2D CNN. By doing so, we can consider the whole volumetric data when spherical surfaces are sampled sufficiently dense, without any information loss. We assessed the proposed method on a publicly available dataset. The experimental evidence shows that the proposed method is effective, outperforming existing methods.

[1]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[2]  Zhenggang Wang and Guanling Wang.  Triplanar Convolutional Neural Network for Automatic Liver and Tumor Image Segmentation , 2018 .

[3]  Seyed-Ahmad Ahmadi,et al.  Automatic Liver and Lesion Segmentation in CT Using Cascaded Fully Convolutional Neural Networks and 3D Conditional Random Fields , 2016, MICCAI.

[4]  Yuichiro Hayashi,et al.  Deep learning and its application to medical image segmentation , 2018, ArXiv.

[5]  Dean C. Barratt,et al.  Automatic Multi-Organ Segmentation on Abdominal CT With Dense V-Networks , 2018, IEEE Transactions on Medical Imaging.

[6]  Trevor Darrell,et al.  Conditional Random Fields for Object Recognition , 2004, NIPS.

[7]  Danilo P. Mandic,et al.  Recurrent Neural Networks for Prediction , 2001 .

[8]  Samy Bengio,et al.  Understanding deep learning requires rethinking generalization , 2016, ICLR.

[9]  Yingwei Li,et al.  Volumetric Medical Image Segmentation: A 3D Deep Coarse-to-Fine Framework and Its Adversarial Examples , 2020, Deep Learning and Convolutional Neural Networks for Medical Imaging and Clinical Informatics.

[10]  Tianwei Ni,et al.  Elastic Boundary Projection for 3D Medical Image Segmentation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Hideki Kakeya,et al.  3D U-JAPA-Net: Mixture of Convolutional Networks for Abdominal Multi-organ CT Segmentation , 2018, MICCAI.

[12]  Le Lu,et al.  Improving Deep Pancreas Segmentation in CT and MRI Images via Recurrent Neural Contextual Learning and Direct Loss Function , 2017, ArXiv.

[13]  William M. Wells,et al.  Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation , 2004, IEEE Transactions on Medical Imaging.

[14]  Zachary Chase Lipton.  A Critical Review of Recurrent Neural Networks for Sequence Learning , 2015, ArXiv.

[15]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[16]  Yoshua Bengio,et al.  A Closer Look at Memorization in Deep Networks , 2017, ICML.

[17]  Maria Wimmer,et al.  Deep Sequential Segmentation of Organs in Volumetric Medical Scans , 2018, IEEE Transactions on Medical Imaging.

[18]  Yan Wang,et al.  Abdominal multi-organ segmentation with organ-attention networks and statistical fusion , 2018, Medical Image Anal..

[19]  Daniel Rueckert,et al.  Computational Anatomy for Multi-Organ Analysis in Medical Imaging: A Review , 2018, Medical Image Anal..

[20]  Danilo P. Mandic,et al.  Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability , 2001 .

[21]  James Bailey,et al.  Dimensionality-Driven Learning with Noisy Labels , 2018, ICML.

[22]  Chi-Wing Fu,et al.  H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes , 2018, IEEE Transactions on Medical Imaging.

[23]  Yuichiro Hayashi,et al.  A multi-scale pyramid of 3D fully convolutional networks for abdominal multi-organ segmentation , 2018, MICCAI.

[24]  Sébastien Ourselin,et al.  Evaluation of Six Registration Methods for the Human Abdomen on Clinically Acquired CT , 2016, IEEE Transactions on Biomedical Engineering.

[25]  Ronald M. Summers,et al.  DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation , 2015, MICCAI.

[26]  Stephen M. Moore,et al.  The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository , 2013, Journal of Digital Imaging.

[27]  Alan L. Yuille,et al.  Bridging the Gap Between 2D and 3D Organ Segmentation , 2018, MICCAI.

[28]  Ronald M. Summers,et al.  Deep convolutional networks for pancreas segmentation in CT imaging , 2015, Medical Imaging.

[29]  Lin Yang,et al.  Combining Fully Convolutional and Recurrent Neural Networks for 3D Biomedical Image Segmentation , 2016, NIPS.

[30]  Stefan Zachow,et al.  Automated segmentation of knee bone and cartilage combining statistical shape knowledge and convolutional neural networks: Data from the Osteoarthritis Initiative , 2019, Medical Image Anal..

[31]  Klaus H. Maier-Hein,et al.  Abstract: nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation , 2019, Bildverarbeitung für die Medizin.

[32]  Vibhav Vineet,et al.  Conditional Random Fields as Recurrent Neural Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).