Crossover-Net: Leveraging vertical-horizontal crossover relation for robust medical image segmentation

Abstract Accurate boundary segmentation in medical images is significant yet challenging due to large variation of shape, size and appearance within intra- and inter- samples. In this paper, we present a novel deep model termed as Crossover-Net for robust segmentation in medical images. The proposed model is inspired by an interesting observation – the features learned from horizontal and vertical directions can provide informative and complement contextual information to enhance discriminative ability between different tissues. Specifically, we first originally propose a cross-shaped patch, namely crossover-patch which consists of a pair of (orthogonal and overlapping) vertical and horizontal patches. Then, we develop our Crossover-Net to learn the vertical and horizontal crossover relation according to the proposed crossover-patches. To train our model end-to-end, we design a novel loss function to (1) impose the consistency on overlapping region of vertical and horizontal patches and (2) preserve the diversity on their non-overlapping regions. We have extensively evaluated our method on CT kidney tumor, MR cardiac, and X-ray breast mass segmentation tasks, showing promising results compared with the current state-of-the-art methods. The code is available at https://github.com/Qianyu1226/Crossover-Net .

[1]  Shubham Jain,et al.  2D-3D Fully Convolutional Neural Networks for Cardiac MR Segmentation , 2017, STACOM@MICCAI.

[2]  Jinjun Xiong,et al.  MSU-Net: Multiscale Statistical U-Net for Real-time 3D Cardiac MRI Video Segmentation , 2019, MICCAI.

[3]  Yinghuan Shi,et al.  Semi-Automatic Segmentation of Prostate in CT Images via Coupled Feature Representation and Spatial-Constrained Transductive Lasso , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Raghav Mehta,et al.  M-net: A Convolutional Neural Network for deep brain structure segmentation , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[5]  Akshay Pai,et al.  One Network to Segment Them All: A General, Lightweight System for Accurate 3D Medical Image Segmentation , 2019, MICCAI.

[6]  Xudong Jiang,et al.  Semantic Correlation Promoted Shape-Variant Context for Segmentation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Yinghuan Shi,et al.  Crossbar-Net: A Novel Convolutional Neural Network for Kidney Tumor Segmentation in CT Images , 2018, IEEE Transactions on Image Processing.

[8]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  M. Viergever,et al.  Automatic Segmentation of MR Brain Images With a Convolutional Neural Network. , 2016, IEEE transactions on medical imaging.

[10]  Ulas Bagci,et al.  Capsules for Object Segmentation , 2018, ArXiv.

[11]  Richard H. Moore,et al.  THE DIGITAL DATABASE FOR SCREENING MAMMOGRAPHY , 2007 .

[12]  Meng Zhou,et al.  Photoacoustic Image Classification and Segmentation of Breast Cancer: A Feasibility Study , 2019, IEEE Access.

[13]  Raunak Dey,et al.  Diagnostic classification of lung nodules using 3D neural networks , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[14]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Ender Konukoglu,et al.  PHiSeg: Capturing Uncertainty in Medical Image Segmentation , 2019, MICCAI.

[16]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[17]  John K. Tsotsos,et al.  Efficient and generalizable statistical models of shape and appearance for analysis of cardiac MRI , 2008, Medical Image Anal..

[18]  Jiasong Wu,et al.  Automatic Segmentation of Kidney and Renal Tumor in CT Images Based on 3D Fully Convolutional Neural Network with Pyramid Pooling Module , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[19]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[20]  Gustavo Carneiro,et al.  Mass segmentation in mammograms: A cross-sensor comparison of deep and tailored features , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[21]  Richard H. Moore,et al.  Current Status of the Digital Database for Screening Mammography , 1998, Digital Mammography / IWDM.

[22]  Yinghuan Shi,et al.  A learning-based CT prostate segmentation method via joint transductive feature selection and regression , 2016, Neurocomputing.

[23]  Dimitris N. Metaxas,et al.  Fully Automatic Segmentation Of Short-Axis Cardiac MRI Using Modified Deep Layer Aggregation , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[24]  Jaime S. Cardoso,et al.  INbreast: toward a full-field digital mammographic database. , 2012, Academic radiology.

[25]  Yan Wang,et al.  Hyper-Pairing Network for Multi-Phase Pancreatic Ductal Adenocarcinoma Segmentation , 2019, MICCAI.

[26]  Gustavo Carneiro,et al.  Deep structured learning for mass segmentation from mammograms , 2014, 2015 IEEE International Conference on Image Processing (ICIP).

[27]  Chao Huang,et al.  3D U2-Net: A 3D Universal U-Net for Multi-Domain Medical Image Segmentation , 2019, MICCAI.

[28]  Chi-Wing Fu,et al.  H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes , 2018, IEEE Transactions on Medical Imaging.

[29]  Yinghuan Shi,et al.  Does Manual Delineation only Provide the Side Information in CT Prostate Segmentation? , 2017, MICCAI.

[30]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[31]  Zhenyu Liu,et al.  Central focused convolutional neural networks: Developing a data-driven model for lung nodule segmentation , 2017, Medical Image Anal..

[32]  Shunxing Bao,et al.  Splenomegaly Segmentation on Multi-Modal MRI Using Deep Convolutional Networks , 2018, IEEE Transactions on Medical Imaging.

[33]  Zhiming Luo,et al.  GridNet with Automatic Shape Prior Registration for Automatic MRI Cardiac Segmentation , 2017, STACOM@MICCAI.

[34]  Muhammad Imran,et al.  Efficient Brain Tumor Segmentation With Multiscale Two-Pathway-Group Conventional Neural Networks , 2019, IEEE Journal of Biomedical and Health Informatics.

[35]  Dinggang Shen,et al.  High-Resolution Encoder–Decoder Networks for Low-Contrast Medical Image Segmentation , 2020, IEEE Transactions on Image Processing.

[36]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[37]  Nassir Navab,et al.  Multi-scale Microaneurysms Segmentation Using Embedding Triplet Loss , 2019, MICCAI.

[38]  Luca Maria Gambardella,et al.  Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images , 2012, NIPS.

[39]  Daniel Rueckert,et al.  Automatic 3D bi-ventricular segmentation of cardiac images by a shape-refined multi-task deep learning approach , 2018, IEEE Transactions on Medical Imaging.

[40]  Christian Wachinger,et al.  DeepNAT: Deep convolutional neural network for segmenting neuroanatomy , 2017, NeuroImage.

[41]  Nico Karssemeijer,et al.  Non-uniform patch sampling with deep convolutional neural networks for white matter hyperintensity segmentation , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[42]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[43]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Shenghua Gao,et al.  CE-Net: Context Encoder Network for 2D Medical Image Segmentation , 2019, IEEE Transactions on Medical Imaging.

[45]  Weidong Cai,et al.  HD-Net: Hybrid Discriminative Network for Prostate Segmentation in MR Images , 2019, MICCAI.

[46]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[47]  Gregory D. Hager,et al.  Adversarial deep structured nets for mass segmentation from mammograms , 2017, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).