Oral-3D: Reconstructing the 3D Bone Structure of Oral Cavity from 2D Panoramic X-ray

Panoramic X-ray and Cone Beam Computed Tomography (CBCT) are two of the most general imaging methods in digital dentistry. While CBCT can provide higher-dimension information, the panoramic X-ray has the advantages of lower radiation dose and cost. Consequently, generating 3D information of bony tissues from the X-ray that can reflect dental diseases is of great interest. This technique can be even more helpful for developing areas where the CBCT is not always available due to the lack of screening machines or high screening cost. In this paper, we present \textit{Oral-3D} to reconstruct the bone structure of oral cavity from a single panoramic X-ray image by taking advantage of some prior knowledge in oral structure, which conventionally can only be obtained by a 3D imaging method like CBCT. Specifically, we first train a generative network to back project the 2D X-ray image into 3D space, then restore the bone structure by registering the generated 3D image with the prior shape of the dental arch. To be noted, \textit{Oral-3D} can restore both the density of bony tissues and the curved mandible surface. Experimental results show that our framework can reconstruct the 3D structure with significantly high quality. To the best of our knowledge, this is the first work that explores 3D reconstruction from a 2D image in dental health.

[1]  S Braun,et al.  The form of the human dental arch. , 2009, The Angle orthodontist.

[2]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[3]  Mila Nikolova,et al.  Algorithms for Finding Global Minimizers of Image Segmentation and Denoising Models , 2006, SIAM J. Appl. Math..

[4]  Sharon L. Brooks,et al.  CBCT Dosimetry: Orthodontic Considerations , 2009 .

[5]  Norihiko Okada,et al.  Diagnostic accuracy of cone-beam CT in the assessment of mandibular invasion of lower gingival carcinoma: comparison with conventional panoramic radiography. , 2009, European journal of radiology.

[6]  Xavier Bresson,et al.  Geometric Applications of the Split Bregman Method: Segmentation and Surface Reconstruction , 2010, J. Sci. Comput..

[7]  Daniel Cremers,et al.  Fast Joint Estimation of Silhouettes and Dense 3D Geometry from Multiple Images , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[9]  Ann Wenzel,et al.  Image and surgery-related costs comparing cone beam CT and panoramic imaging before removal of impacted mandibular third molars. , 2014, Dento maxillo facial radiology.

[10]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[11]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[12]  Silvio Savarese,et al.  3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction , 2016, ECCV.

[13]  Sultan Imangaliyev,et al.  Deep Learning for Classification of Dental Plaque Images , 2016, MOD.

[14]  R. Nagaraj,et al.  Classification of dental diseases using CNN and transfer learning , 2017, 2017 5th International Symposium on Computational and Business Intelligence (ISCBI).

[15]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Raymond Y. K. Lau,et al.  Least Squares Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[17]  Pedro Costa,et al.  Towards Adversarial Retinal Image Synthesis , 2017, ArXiv.

[18]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[20]  Dong Soo Lee,et al.  Generation of Structural MR Images from Amyloid PET: Application to MR-Less Quantification , 2017, The Journal of Nuclear Medicine.

[21]  Jae‐Hong Lee,et al.  Detection and diagnosis of dental caries using a deep learning-based convolutional neural network algorithm. , 2018, Journal of dentistry.

[22]  Timo Ropinski,et al.  Single‐image Tomography: 3D Volumes from 2D Cranial X‐Rays , 2017, Comput. Graph. Forum.

[23]  Clara Fernandez-Labrador,et al.  Layouts From Panoramic Images With Geometry and Deep Learning , 2018, IEEE Robotics and Automation Letters.

[24]  Serge J. Belongie,et al.  Learning Single-View 3D Reconstruction with Limited Pose Supervision , 2018, ECCV.

[25]  Jiajun Wu,et al.  Learning Shape Priors for Single-View 3D Completion and Reconstruction , 2018, ECCV.

[26]  Xuenan Cui,et al.  Deep CT to MR Synthesis Using Paired and Unpaired Data , 2018, Sensors.

[27]  Zhiming Cui,et al.  ToothNet: Automatic Tooth Instance Segmentation and Identification From Cone Beam CT Images , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Kai Ma,et al.  X2CT-GAN: Reconstructing CT From Biplanar X-Rays With Generative Adversarial Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Qianjin Feng,et al.  Automatic reconstruction method for high-contrast panoramic image from dental cone-beam CT data , 2019, Comput. Methods Programs Biomed..

[30]  Adrian Basarab,et al.  Deep Learning-Based Super-Resolution Applied to Dental Computed Tomography , 2019, IEEE Transactions on Radiation and Plasma Medical Sciences.