Organ At Risk Segmentation with Multiple Modality

With the development of image segmentation in computer vision, biomedical image segmentation have achieved remarkable progress on brain tumor segmentation and Organ At Risk (OAR) segmentation. However, most of the research only uses single modality such as Computed Tomography (CT) scans while in real world scenario doctors often use multiple modalities to get more accurate result. To better leverage different modalities, we have collected a large dataset consists of 136 cases with CT and MR images which diagnosed with nasopharyngeal cancer. In this paper, we propose to use Generative Adversarial Network to perform CT to MR transformation to synthesize MR images instead of aligning two modalities. The synthesized MR can be jointly trained with CT to achieve better performance. In addition, we use instance segmentation model to extend the OAR segmentation task to segment both organs and tumor region. The collected dataset will be made public soon.

[1]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Ronan Collobert,et al.  Learning to Segment Object Candidates , 2015, NIPS.

[3]  Gabriele Lombardi,et al.  Automatic Abdominal Organ Segmentation from CT images , 2009 .

[4]  Zhe Guo,et al.  Medical image segmentation based on multi-modal convolutional neural network: Study on image fusion schemes , 2017, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[5]  Vincent Dumoulin,et al.  Deconvolution and Checkerboard Artifacts , 2016 .

[6]  Luc Van Gool,et al.  Pose Guided Person Image Generation , 2017, NIPS.

[7]  Léon Bottou,et al.  Wasserstein GAN , 2017, ArXiv.

[8]  I. El Naqa,et al.  A radiomics model from joint FDG-PET and MRI texture features for the prediction of lung metastases in soft-tissue sarcomas of the extremities , 2015, Physics in medicine and biology.

[9]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[10]  Parashkev Nachev,et al.  Computer Methods and Programs in Biomedicine NiftyNet: a deep-learning platform for medical imaging , 2022 .

[11]  F. Turkheimer,et al.  A radiomics model from joint FDG-PET and MRI texture features for the prediction of lung metastases in soft-tissue sarcomas of the extremities , 2015 .

[12]  Jelmer M. Wolterink,et al.  Deep MR to CT Synthesis Using Unpaired Data , 2017, SASHIMI@MICCAI.

[13]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[14]  Bulat Ibragimov,et al.  Segmentation of organs‐at‐risks in head and neck CT images using convolutional neural networks , 2017, Medical physics.

[15]  Faisal Mahmood,et al.  Unsupervised Reverse Domain Adaptation for Synthetic Medical Images via Adversarial Training , 2017, IEEE Transactions on Medical Imaging.

[16]  Ronan Collobert,et al.  Learning to Refine Object Segments , 2016, ECCV.

[17]  Yi Li,et al.  Fully Convolutional Instance-Aware Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Sébastien Ourselin,et al.  Automatic Brain Tumor Segmentation Using Cascaded Anisotropic Convolutional Neural Networks , 2017, BrainLes@MICCAI.

[19]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Su Ruan,et al.  Medical Image Synthesis with Context-Aware Generative Adversarial Networks , 2016, MICCAI.

[21]  Kaiming He,et al.  Mask R-CNN , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[22]  Dinggang Shen,et al.  Segmentation of Organs at Risk in thoracic CT images using a SharpMask architecture and Conditional Random Fields , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[23]  Sébastien Ourselin,et al.  Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations , 2017, DLMIA/ML-CDS@MICCAI.

[24]  Rob Fergus,et al.  Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[25]  Sotirios A. Tsaftaris,et al.  Adversarial Image Synthesis for Unpaired Multi-modal Cardiac Data , 2017, SASHIMI@MICCAI.

[26]  M van Herk,et al.  Magnetic resonance image-directed stereotactic neurosurgery: use of image fusion with computerized tomography to enhance spatial accuracy. , 1995, Journal of neurosurgery.

[27]  Dumitru Erhan,et al.  Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  David Berthelot,et al.  BEGAN: Boundary Equilibrium Generative Adversarial Networks , 2017, ArXiv.

[29]  Dinggang Shen,et al.  Joint Segmentation of Multiple Thoracic Organs in CT Images with Two Collaborative Deep Architectures , 2017, DLMIA/ML-CDS@MICCAI.

[30]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[32]  Cheng Huang,et al.  Fully Automatic Multi-Organ Segmentation Based on Multi-Boost Learning and Statistical Shape Model Search , 2015, VISCERAL Challenge@ISBI.

[33]  Konstantinos Kamnitsas,et al.  Unsupervised domain adaptation in brain lesion segmentation with adversarial networks , 2016, IPMI.

[34]  Dean C. Barratt,et al.  Automatic Multi-Organ Segmentation on Abdominal CT With Dense V-Networks , 2018, IEEE Transactions on Medical Imaging.

[35]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[36]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.