3D Probabilistic Segmentation and Volumetry from 2D projection images

X-Ray imaging is quick, cheap and useful for front-line care assessment and intra-operative real-time imaging (e.g., C-Arm Fluoroscopy). However, it suffers from projective information loss and lacks vital volumetric information on which many essential diagnostic biomarkers are based on. In this paper we explore probabilistic methods to reconstruct 3D volumetric images from 2D imaging modalities and measure the models' performance and confidence. We show our models' performance on large connected structures and we test for limitations regarding fine structures and image domain sensitivity. We utilize fast end-to-end training of a 2D-3D convolutional networks, evaluate our method on 117 CT scans segmenting 3D structures from digitally reconstructed radiographs (DRRs) with a Dice score of $0.91 \pm 0.0013$. Source code will be made available by the time of the conference.

[1]  Jiajun Wu,et al.  Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[2]  Chen Kong,et al.  Learning Efficient Point Cloud Generation for Dense 3D Object Reconstruction , 2017, AAAI.

[3]  Wafa Skalli,et al.  3D reconstruction of rib cage geometry from biplanar radiographs using a statistical parametric model approach , 2016, Comput. methods Biomech. Biomed. Eng. Imaging Vis..

[4]  Alexei A. Efros,et al.  Unsupervised Domain Adaptation through Self-Supervision , 2019, ArXiv.

[5]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[6]  Bishesh Khanal,et al.  Confident Head Circumference Measurement from Ultrasound with Real-time Feedback for Sonographers , 2019, MICCAI.

[7]  Honglak Lee,et al.  Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision , 2016, NIPS.

[8]  Ronald M. Summers,et al.  ChestX-ray: Hospital-Scale Chest X-ray Database and Benchmarks on Weakly Supervised Classification and Localization of Common Thorax Diseases , 2019, Deep Learning and Convolutional Neural Networks for Medical Imaging and Clinical Informatics.

[9]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[10]  Nassir Navab,et al.  X-Ray In-Depth Decomposition: Revealing the Latent Structures , 2016, MICCAI.

[11]  Ender Konukoglu,et al.  PHiSeg: Capturing Uncertainty in Medical Image Segmentation , 2019, MICCAI.

[12]  Kai Ma,et al.  X2CT-GAN: Reconstructing CT From Biplanar X-Rays With Generative Adversarial Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Daniel Rueckert,et al.  High-resolution contrast enhanced multi-phase hepatic Computed Tomography data fromaporcine Radio-Frequency Ablation study , 2014, 2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI).

[14]  P. Van Dyck,et al.  Imaging of pulmonary tuberculosis , 2003, European Radiology.

[15]  Geoffrey D Rubin,et al.  Computed tomography: revolutionizing the practice of medicine for 40 years. , 2014, Radiology.

[16]  Timo Ropinski,et al.  Single‐image Tomography: 3D Volumes from 2D Cranial X‐Rays , 2017, Comput. Graph. Forum.

[17]  Thomas Wischgoll,et al.  Knowledge-Assisted Reconstruction of the Human Rib Cage and Lungs , 2010, IEEE Computer Graphics and Applications.

[18]  Charl P. Botha,et al.  Exposure Render: An Interactive Photo-Realistic Volume Rendering Framework , 2012, PloS one.

[19]  Francesco Visin,et al.  A guide to convolution arithmetic for deep learning , 2016, ArXiv.