Longitudinal Prediction of Infant Diffusion MRI Data via Graph Convolutional Adversarial Networks

Missing data is a common problem in longitudinal studies due to subject dropouts and failed scans. We present a graph-based convolutional neural network to predict missing diffusion MRI data. In particular, we consider the relationships between sampling points in the spatial domain and the diffusion wave-vector domain to construct a graph. We then use a graph convolutional network to learn the non-linear mapping from available data to missing data. Our method harnesses a multi-scale residual architecture with adversarial learning for prediction with greater accuracy and perceptual quality. Experimental results show that our method is accurate and robust in the longitudinal prediction of infant brain diffusion MRI data.

[1]  Nathan D. Cahill,et al.  Robust Spatial Filtering With Graph Convolutional Neural Networks , 2017, IEEE Journal of Selected Topics in Signal Processing.

[2]  Dinggang Shen,et al.  Medical Image Synthesis with Deep Convolutional Adversarial Networks , 2018, IEEE Transactions on Biomedical Engineering.

[3]  Yaozong Gao,et al.  Can we predict subject‐specific dynamic cortical thickness maps during infancy from birth? , 2017, Human brain mapping.

[4]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[5]  P. F. Buckley.  Prenatal and Neonatal Brain Structure and White Matter Maturation in Children at High Risk for Schizophrenia , 2012 .

[6]  Ke Li,et al.  Improving Estimation of Fiber Orientations in Diffusion MRI Using Inter-Subject Information Sharing , 2016, Scientific Reports.

[7]  Richard A. Jones,et al.  MRI of the neonatal brain: optimization of spin-echo parameters. , 2004, AJR. American journal of roentgenology.

[8]  Pierre Vandergheynst,et al.  Geometric Deep Learning: Going beyond Euclidean data , 2016, IEEE Signal Process. Mag..

[9]  Raquel Urtasun,et al.  Understanding the Effective Receptive Field in Deep Convolutional Neural Networks , 2016, NIPS.

[10]  Dinggang Shen,et al.  Neighborhood Matching for Curved Domains with Application to Denoising in Diffusion MRI , 2017, MICCAI.

[11]  Joan Bruna,et al.  Spectral Networks and Locally Connected Networks on Graphs , 2013, ICLR.

[12]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Xavier Bresson,et al.  CayleyNets: Graph Convolutional Neural Networks With Complex Rational Spectral Filters , 2017, IEEE Transactions on Signal Processing.

[14]  Max A. Viergever,et al.  Generative Adversarial Networks for Noise Reduction in Low-Dose CT , 2017, IEEE Transactions on Medical Imaging.

[15]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[16]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[17]  Inderjit S. Dhillon,et al.  Weighted Graph Cuts without Eigenvectors A Multilevel Approach , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Yogesh Rathi,et al.  Spatially Regularized Compressed Sensing for High Angular Resolution Diffusion Imaging , 2011, IEEE Transactions on Medical Imaging.

[19]  Daphne Koller,et al.  Self-Paced Learning for Latent Variable Models , 2010, NIPS.

[20]  Dinggang Shen,et al.  BIRNet: Brain image registration using dual‐supervised fully convolutional networks , 2018, Medical Image Anal..

[21]  Dinggang Shen,et al.  Prediction of Infant MRI Appearance and Anatomical Structure Evolution Using Sparse Patch-Based Metamorphosis Learning Framework , 2015, Patch-MI@MICCAI.

[22]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[23]  François-Xavier Vialard,et al.  Geodesic Regression for Image Time-Series , 2011, MICCAI.

[24]  J. Gilmore,et al.  Mapping region-specific longitudinal cortical surface expansion from birth to 2 years of age. , 2013, Cerebral cortex.

[25]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[26]  M. Jorge Cardoso,et al.  Robust training of recurrent neural networks to handle missing data for disease progression modeling , 2018, ArXiv.

[27]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Rob Fergus,et al.  Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks , 2015, NIPS.

[29]  Abhinav Gupta,et al.  Training Region-Based Object Detectors with Online Hard Example Mining , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[31]  Ehsan Adeli,et al.  3-D Fully Convolutional Networks for Multimodal Isointense Infant Brain Image Segmentation , 2019, IEEE Transactions on Cybernetics.

[32]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[33]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[34]  Ling Shao,et al.  Simultaneous Super-Resolution and Cross-Modality Synthesis of 3D Medical Images Using Weakly-Supervised Joint Convolutional Sparse Coding , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Dinggang Shen,et al.  Computational neuroanatomy of baby brains: A review , 2019, NeuroImage.

[36]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[37]  Dinggang Shen,et al.  Graph-Constrained Sparse Construction of Longitudinal Diffusion-Weighted Infant Atlases , 2017, MICCAI.

[38]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[39]  Joan Bruna,et al.  Deep Convolutional Networks on Graph-Structured Data , 2015, ArXiv.