Ultimate Reconstruction: Understand Your Bones From Orthogonal Views

3D image reconstruction is a common basis of medical image analysis, which requires a sequence of 2D slices/tomograms obtained from the relative motion to provide enough 3D information. When considering only the task to localize exception objects, a pair of two-view perspective 2D images may also be able to provide enough 3D information, which, however, has not been well studied. In this paper, we proposed the concept of Ultimate Reconstruction (UR) that reconstructs a 3D image from only a pair of two-view perspective 2D images. We resort techniques of generative adversarial network (GAN) to deal with this task, where we propose the Sense-consistency GAN (SGAN) with the sense-consistency constraint to learning the potential coarse-to-fine sense information during training the generative model. Experiments on the KiTS19 dataset with 300 subjects demonstrate that our SGAN achieves MAE/SSIM / PSNR values of 11.16% / 66.50%/23.82 when using only two 2D perspective images. It supports the possibility of UR and indicates that SGAN is promising to deal with UR.

[1]  Quanzheng Li,et al.  Iterative PET Image Reconstruction Using Convolutional Neural Network Representation , 2017, IEEE Transactions on Medical Imaging.

[2]  Ruey-Feng Chang,et al.  Tumor Detection in Automated Breast Ultrasound Using 3-D CNN and Prioritized Candidate Aggregation , 2019, IEEE Transactions on Medical Imaging.

[3]  Michael Unser,et al.  CNN-Based Projected Gradient Descent for Consistent CT Image Reconstruction , 2017, IEEE Transactions on Medical Imaging.

[4]  Chunfeng Lian,et al.  Spatially-Constrained Fisher Representation for Brain Disease Identification With Incomplete Multi-Modal Neuroimages , 2020, IEEE Transactions on Medical Imaging.

[5]  Daniel Rueckert,et al.  Convolutional Recurrent Neural Networks for Dynamic MR Image Reconstruction , 2017, IEEE Transactions on Medical Imaging.

[6]  Abdullah Abusorrah,et al.  Effective Visual Domain Adaptation via Generative Adversarial Distribution Matching , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[7]  Scott A. Banks,et al.  Automated Registration of 3-D Knee Implant Models to Fluoroscopic Images Using Lipschitzian Optimization , 2018, IEEE Transactions on Medical Imaging.

[8]  J. Wardlaw,et al.  The detection and management of unruptured intracranial aneurysms. , 2000, Brain : a journal of neurology.

[9]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Konstantinos Kamnitsas,et al.  Generative adversarial networks and adversarial methods in biomedical image analysis , 2018, ArXiv.

[11]  Xuanqin Mou,et al.  Low-Dose CT Image Denoising Using a Generative Adversarial Network With Wasserstein Distance and Perceptual Loss , 2017, IEEE Transactions on Medical Imaging.