A deep network for tissue microstructure estimation using modified LSTM units

Diffusion magnetic resonance imaging (dMRI) offers a unique tool for noninvasively assessing tissue microstructure. However, accurate estimation of tissue microstructure described by complicated signal models can be challenging when a reduced number of diffusion gradients are used. Deep learning based microstructure estimation has recently been developed and achieved promising results. In particular, optimization-based learning, where deep network structures are constructed by unfolding the iterative processes performed for solving optimization problems, has demonstrated great potential in accurate microstructure estimation with a reduced number of diffusion gradients. In this work, using the optimization-based learning strategy, we propose a deep network structure that is motivated by the use of historical information in iterative optimization for tissue microstructure estimation, and such incorporation of historical information has not been previously explored in the design of deep networks for microstructure estimation. We assume that (1) diffusion signals can be sparsely represented by a dictionary and its coefficients jointly in the spatial and angular domain, and (2) tissue microstructure can be computed from the sparse representation. Following these assumptions, our network comprises two cascaded stages. The first stage takes image patches as input and computes the spatial-angular sparse representation of the input with learned weights. Specifically, the network structure in the first stage is constructed by unfolding an iterative process for solving sparse reconstruction problems, where historical information is incorporated. The components in this network can be shown to correspond to modified long short-term memory (LSTM) units. In the second stage, fully connected layers are added to compute the mapping from the sparse representation to tissue microstructure. The weights in the two stages are learned jointly by minimizing the mean squared error of microstructure estimation. Experiments were performed on dMRI scans with a reduced number of diffusion gradients. For demonstration, we evaluated the estimation of tissue microstructure described by three signal models: the neurite orientation dispersion and density imaging (NODDI) model, the spherical mean technique (SMT) model, and the ensemble average propagator (EAP) model. The results indicate that the proposed approach outperforms competing methods.

[1]  O. Güntürkün,et al.  Neurite architecture of the planum temporale predicts neurophysiological processing of auditory speech , 2018, Science Advances.

[2]  Yann LeCun,et al.  Learning Fast Approximations of Sparse Coding , 2010, ICML.

[3]  Antonio Criminisi,et al.  Image quality transfer and applications in diffusion MRI , 2017, NeuroImage.

[4]  Antonio Criminisi,et al.  Bayesian Image Quality Transfer with CNNs: Exploring Uncertainty in dMRI Super-Resolution , 2017, MICCAI.

[5]  Jerry L. Prince,et al.  Estimation of fiber orientations using neighborhood information , 2016, Medical Image Anal..

[6]  Jerry L. Prince,et al.  Resolution of crossing fibers with constrained compressed sensing using diffusion tensor MRI , 2012, NeuroImage.

[7]  Emily C. Collins,et al.  Application of neurite orientation dispersion and density imaging (NODDI) to a tau pathology model of Alzheimer's disease , 2016, NeuroImage.

[8]  Chuyang Ye Estimation of Tissue Microstructure Using a Deep Network Inspired by a Sparse Reconstruction Framework , 2017, IPMI.

[9]  Thomas W. McAllister,et al.  Hybrid Diffusion Imaging in Mild Traumatic Brain Injury , 2018, Journal of neurotrauma.

[10]  Richard Bowtell,et al.  Microstructural imaging of the human brain with a ‘super-scanner’: 10 key advantages of ultra-strong gradients for diffusion MRI , 2018, NeuroImage.

[11]  Antonio M. López,et al.  The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  J. L. Conel,et al.  The postnatal development of the human cerebral cortex , 1960 .

[13]  Daniel C. Alexander,et al.  NODDI: Practical in vivo neurite orientation dispersion and density imaging of the human brain , 2012, NeuroImage.

[14]  Jerry L. Prince,et al.  Fiber Orientation Estimation Guided by a Deep Network , 2017, MICCAI.

[15]  Ed X. Wu,et al.  Towards better MR characterization of neural tissues using directional diffusion kurtosis analysis , 2008, NeuroImage.

[16]  Rachid Deriche,et al.  A computational diffusion MRI and parametric dictionary learning framework for modeling the diffusion signal and its features , 2013, Medical Image Anal..

[17]  Bernhard Schölkopf,et al.  A Kernel Two-Sample Test , 2012, J. Mach. Learn. Res..

[18]  Jelle Veraart,et al.  One diffusion acquisition and different white matter models: How does microstructure change in human early development based on WMTI and NODDI? , 2015, NeuroImage.

[19]  Nick C Fox,et al.  Cortical microstructure in young onset Alzheimer's disease using neurite orientation dispersion and density imaging , 2018, Human brain mapping.

[20]  Ning Qian,et al.  On the momentum term in gradient descent learning algorithms , 1999, Neural Networks.

[21]  René Vidal,et al.  Joint spatial‐angular sparse coding for dMRI with separable dictionaries , 2018, Medical Image Anal..

[22]  Deanne K. Thompson,et al.  Axon density and axon orientation dispersion in children born preterm , 2016, Human brain mapping.

[23]  Essa Yacoub,et al.  The WU-Minn Human Connectome Project: An overview , 2013, NeuroImage.

[24]  Qing Ling,et al.  Learning deep l0 encoders , 2016, AAAI 2016.

[25]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[26]  Timothy Edward John Behrens,et al.  Diffusion MRI : from quantitative measurement to in vivo neuroanatomy , 2014 .

[27]  René Vidal,et al.  Spatial-Angular Sparse Coding for HARDI , 2016, MICCAI.

[28]  Osamu Abe,et al.  Neurite orientation dispersion and density imaging in the substantia nigra in idiopathic Parkinson disease , 2016, European Radiology.

[29]  Craig A Branch,et al.  Diffusion-tensor imaging implicates prefrontal axonal injury in executive function impairment following very mild traumatic brain injury. , 2009, Radiology.

[30]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[31]  Carlo Pierpaoli,et al.  Mean apparent propagator (MAP) MRI: A novel diffusion imaging method for mapping tissue microstructure , 2013, NeuroImage.

[32]  Wen Gao,et al.  Maximal Sparsity with Deep Networks? , 2016, NIPS.

[33]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[34]  Guillermo Sapiro,et al.  Learning Efficient Sparse and Low Rank Models , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  A. Dale,et al.  Clinical findings and white matter abnormalities seen on diffusion tensor imaging in adolescents with very low birth weight. , 2007, Brain : a journal of neurology.

[36]  Rex E. Jung,et al.  Diffusion markers of dendritic density and arborization in gray matter predict differences in intelligence , 2018, Nature Communications.

[37]  Bailey A. Box,et al.  Application and evaluation of NODDI in the cervical spinal cord of multiple sclerosis patients , 2017, NeuroImage: Clinical.

[38]  Vincent Dousset,et al.  Deciphering the microstructure of hippocampal subfields with in vivo DTI and NODDI: Applications to experimental multiple sclerosis , 2018, NeuroImage.

[39]  Maxime Descoteaux,et al.  Dipy, a library for the analysis of diffusion MRI data , 2014, Front. Neuroinform..

[40]  Bernard Ghanem,et al.  ISTA-Net: Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[41]  Chiara Nosarti,et al.  Early development of structural networks and the impact of prematurity on brain connectivity , 2017, NeuroImage.

[42]  G. Barker,et al.  Diffusion tensor imaging of lesions and normal-appearing white matter in multiple sclerosis , 1999, Neurology.

[43]  Rachid Deriche,et al.  Continuous diffusion signal, EAP and ODF estimation via Compressive Sensing in diffusion MRI , 2013, Medical Image Anal..

[44]  Justin Domke,et al.  Generic Methods for Optimization-Based Modeling , 2012, AISTATS.

[45]  Chuyang Ye,et al.  Tissue microstructure estimation using a deep network inspired by a dictionary‐based framework , 2017, Medical Image Anal..

[46]  Jian Sun,et al.  Deep ADMM-Net for Compressive Sensing MRI , 2016, NIPS.

[47]  Andrew L. Alexander,et al.  Computation of Diffusion Function Measures in $q$ -Space Using Magnetic Resonance Hybrid Diffusion Imaging , 2008, IEEE Transactions on Medical Imaging.

[48]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[49]  Nikos Evangelou,et al.  Quantitative pathological evidence for axonal loss in normal appearing white matter in multiple sclerosis , 2000, Annals of neurology.

[50]  Hui Zhang,et al.  Advanced diffusion imaging sequences could aid assessing patients with focal cortical dysplasia and epilepsy☆ , 2014, Epilepsy Research.

[51]  Kristen M Harris,et al.  Dendritic Spine Pathology: Cause or Consequence of Neurological Disorders? , 2002, Brain Research Reviews.

[52]  Martha Elizabeth Shenton,et al.  Advances in microstructural diffusion neuroimaging for psychiatric disorders , 2018, NeuroImage.

[53]  Hui Zhang,et al.  Imaging brain microstructure with diffusion MRI: practicality and applications , 2019, NMR in biomedicine.

[54]  Daniel C. Alexander,et al.  Multi-compartment microscopic diffusion imaging , 2016, NeuroImage.

[55]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[56]  Hui Zhang,et al.  Assessing white matter microstructure of the newborn with multi-shell diffusion MRI and biophysical compartment models , 2014, NeuroImage.

[57]  Carl-Fredrik Westin,et al.  Estimating Diffusion Propagator and Its Moments Using Directional Radial Basis Functions , 2015, IEEE Transactions on Medical Imaging.

[58]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[59]  L. Lotspeich,et al.  White matter structure in autism: preliminary evidence from diffusion tensor imaging , 2004, Biological Psychiatry.

[60]  T. Blumensath,et al.  Iterative Thresholding for Sparse Approximations , 2008 .

[61]  Qing Ling,et al.  Learning a deep l ∞ encoder for hashing , 2016, IJCAI 2016.

[62]  Daniel Cremers,et al.  q-Space Deep Learning: Twelve-Fold Shorter and Model-Free Diffusion MRI Scans , 2016, IEEE Transactions on Medical Imaging.

[63]  P. Basser,et al.  MR diffusion tensor spectroscopy and imaging. , 1994, Biophysical journal.

[64]  Joseph A. Helpern,et al.  Modeling white matter microstructure with fiber ball imaging , 2018, NeuroImage.

[65]  T. Georgiou,et al.  Microstructure Imaging of Crossing (MIX) White Matter Fibers from diffusion MRI , 2016, Scientific Reports.

[66]  Ivor W. Tsang,et al.  SC2Net: Sparse LSTMs for Sparse Coding , 2018, AAAI.

[67]  Léon Bottou,et al.  Wasserstein Generative Adversarial Networks , 2017, ICML.

[68]  Jean-Philippe Thiran,et al.  Accelerated Microstructure Imaging via Convex Optimization (AMICO) from diffusion MRI data , 2015, NeuroImage.

[69]  Rachid Deriche,et al.  Mipy: An Open-Source Framework to improve reproducibility in Brain Microstructure Imaging , 2018 .

[70]  Noboru Murata,et al.  Neural Network with Unbounded Activation Functions is Universal Approximator , 2015, 1505.03654.

[71]  Scott Holland,et al.  Neurite density index is sensitive to age related differences in the developing brain , 2017, NeuroImage.

[72]  Jean-Philippe Thiran,et al.  Connectivity and tissue microstructural alterations in right and left temporal lobe epilepsy revealed by diffusion spectrum imaging , 2014, NeuroImage: Clinical.

[73]  Peter Savadjiev,et al.  Inter-site and inter-scanner diffusion MRI data harmonization , 2016, NeuroImage.

[74]  Joseph V. Hajnal,et al.  Different patterns of cortical maturation before and after 38 weeks gestational age demonstrated by diffusion MRI in vivo , 2019, NeuroImage.

[75]  F. Kruggel,et al.  Quantitative mapping of the per‐axon diffusion coefficients in brain white matter , 2015, Magnetic resonance in medicine.

[76]  David P. Wipf,et al.  From Bayesian Sparsity to Gated Recurrent Nets , 2017, NIPS.

[77]  B. Jacobs,et al.  Life‐span dendritic and spine changes in areas 10 and 18 of human cortex: A quantitative golgi study , 1997, The Journal of comparative neurology.