AnatomyGen: Deep Anatomy Generation From Dense Representation With Applications in Mandible Synthesis

This work is an effort in human anatomy synthesis using deep models. Here, we introduce a deterministic deep convolutional architecture to generate human anatomies represented as 3D binarized occupancy maps (voxel-grids). The shape generation process is constrained by the 3D coordinates of a small set of landmarks selected on the surface of the anatomy. The proposed learning framework is empirically tested on the mandible bone where it was able to reconstruct the anatomies from landmark coordinates with the average landmark-to-surface error of 1.42 mm. Moreover, the model was able to linearly interpolate in the Z-space and smoothly morph a given 3D anatomy to another. The proposed approach can potentially be used in semi-automated segmentation with manual landmark selection as well as biomechanical modeling. Our main contribution is to demonstrate that deep convolutional architectures can generate high fidelity complex human anatomies from abstract representations.

[1]  Thomas A. Funkhouser,et al.  Interactive 3D Modeling with a Generative Adversarial Network , 2017, 2017 International Conference on 3D Vision (3DV).

[2]  Jiajun Wu,et al.  Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling , 2016, NIPS.

[3]  Polina Golland,et al.  Statistical shape analysis: From landmarks to diffeomorphisms , 2016, Medical Image Anal..

[4]  Silvio Savarese,et al.  3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction , 2016, ECCV.

[5]  Jan Egger,et al.  Mandibular CT Dataset Collection , 2018 .

[6]  Theodore Lim,et al.  Generative and Discriminative Voxel Modeling with Convolutional Neural Networks , 2016, ArXiv.

[7]  Abhinav Gupta,et al.  Learning a Predictable and Generative Vector Representation for Objects , 2016, ECCV.

[8]  Jiajun Wu,et al.  Synthesizing 3D Shapes via Modeling Multi-view Depth Maps and Silhouettes with Deep Generative Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Simon J. Julier,et al.  Structured Prediction of Unobserved Voxels from a Single Depth Image , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Dinggang Shen,et al.  Synthesizing Missing PET from MRI with Cycle-consistent Generative Adversarial Networks for Alzheimer's Disease Diagnosis , 2018, MICCAI.

[11]  Shakir Mohamed,et al.  Variational Approaches for Auto-Encoding Generative Adversarial Networks , 2017, ArXiv.

[12]  Thomas Brox,et al.  Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[13]  Carl Kesselman,et al.  The FaceBase Consortium: a comprehensive resource for craniofacial researchers , 2016, Development.

[14]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[15]  Alejandro F. Frangi,et al.  A Multi-resolution T-Mixture Model Approach to Robust Group-Wise Alignment of Shapes , 2016, MICCAI.

[16]  Blake Hannaford,et al.  Evaluation of segmentation methods on head and neck CT: Auto‐segmentation challenge 2015 , 2017, Medical physics.

[17]  Nassir Navab,et al.  GANs for Medical Image Analysis , 2018, Artif. Intell. Medicine.

[18]  Ajay Seth,et al.  Is my model good enough? Best practices for verification and validation of musculoskeletal models and simulations of movement. , 2015, Journal of biomechanical engineering.

[19]  Konstantinos Kamnitsas,et al.  Anatomically Constrained Neural Networks (ACNNs): Application to Cardiac Image Enhancement and Segmentation , 2017, IEEE Transactions on Medical Imaging.

[20]  David Meger,et al.  Improved Adversarial Systems for 3D Object Generation and Reconstruction , 2017, CoRL.

[21]  Michael Damsgaard,et al.  Introduction to Force-Dependent Kinematics: Theory and Application to Mandible Modeling. , 2017, Journal of biomechanical engineering.

[22]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[24]  Nassir Navab,et al.  Generating Highly Realistic Images of Skin Lesions with GANs , 2018, OR 2.0/CARE/CLIP/ISIC@MICCAI.