Ensemble of expert deep neural networks for spatio‐temporal denoising of contrast‐enhanced MRI sequences

HighlightsWe present a novel spatio‐temporal denoising framework, based on Deep Neural Networks (DNNs), for the analysis of DCE‐MRI sequences of the brain.Specifically, DCE‐MRI denoising facilitates the extraction of the pharmacokinetic (PK) parameters from the DCE‐MRI concentration curves (CTCs) and thus allows a quantitative assessment of blood‐brain barrier functionality.This is accomplished by an ensemble of expert DNNs (deep‐autoencoders), where each is trained on a specific subset of the input space to accommodate different noise characteristics and curve prototypes.We introduce a novel sampling scheme, for generating realistic training sets that faithfully model the DCE‐MRI data and accounts for statistical similarity of neighboring CTCs.Experimental results on synthesized realistic data demonstrate that PK parameters reconstruction is more accurate for denoised CTCs using our method in comparison to CTCs denoised by other methods.The proposed approach is also applied to real DCE‐MRI scans from patients with stroke and brain tumors and is shown to favorably compare to state‐of‐the‐art denoising methods. Graphical abstract Figure. No caption available. ABSTRACT Dynamic contrast‐enhanced MRI (DCE‐MRI) is an imaging protocol where MRI scans are acquired repetitively throughout the injection of a contrast agent. The analysis of dynamic scans is widely used for the detection and quantification of blood‐brain barrier (BBB) permeability. Extraction of the pharmacokinetic (PK) parameters from the DCE‐MRI concentration curves allows quantitative assessment of the integrity of the BBB functionality. However, curve fitting required for the analysis of DCE‐MRI data is error‐prone as the dynamic scans are subject to non‐white, spatially‐dependent and anisotropic noise. We present a novel spatio‐temporal framework based on Deep Neural Networks (DNNs) to address the DCE‐MRI denoising challenges. This is accomplished by an ensemble of expert DNNs constructed as deep autoencoders, where each is trained on a specific subset of the input space to accommodate different noise characteristics and curve prototypes. Spatial dependencies of the PK dynamics are captured by incorporating the curves of neighboring voxels in the entire process. The most likely reconstructed curves are then chosen using a classifier DNN followed by a quadratic programming optimization. As clean signals (ground‐truth) for training are not available, a fully automatic model for generating realistic training sets with complex nonlinear dynamics is introduced. The proposed approach has been successfully applied to full and even temporally down‐sampled DCE‐MRI sequences, from two different databases, of stroke and brain tumor patients, and is shown to favorably compare to state‐of‐the‐art denoising methods.

[1]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[2]  I. J. Schoenberg Cardinal Spline Interpolation , 1987 .

[3]  Kenya Murase,et al.  Efficient method for calculating kinetic parameters using T1‐weighted dynamic contrast‐enhanced magnetic resonance imaging , 2004, Magnetic resonance in medicine.

[4]  Sajan Goud Lingala,et al.  Highly Accelerated Brain DCE MRI with Direct Estimation of Pharmacokinetic Parameter Maps , 2014 .

[5]  Shaohui Liu,et al.  Medical image denoising using convolutional neural network: a residual learning approach , 2017, The Journal of Supercomputing.

[6]  Michael Brady,et al.  Improved Optimization for the Robust and Accurate Linear Registration and Motion Correction of Brain Images , 2002, NeuroImage.

[7]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[8]  Jean-Michel Morel,et al.  Nonlocal Image and Movie Denoising , 2008, International Journal of Computer Vision.

[9]  Alon Friedman,et al.  Blood–brain barrier breakdown as a therapeutic target in traumatic brain injury , 2010, Nature Reviews Neurology.

[10]  Walter H. Backes,et al.  Neurovascular unit impairment in early Alzheimer's disease measured with magnetic resonance imaging , 2016, Neurobiology of Aging.

[11]  Daniel Rueckert,et al.  Medical Image Computing and Computer-Assisted Intervention − MICCAI 2017: 20th International Conference, Quebec City, QC, Canada, September 11-13, 2017, Proceedings, Part II , 2017, Lecture Notes in Computer Science.

[12]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[13]  Steven P Sourbron,et al.  Classic models for dynamic contrast‐enhanced MRI , 2013, NMR in biomedicine.

[14]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups , 2012, IEEE Signal Processing Magazine.

[15]  L R Schad,et al.  Pharmacokinetic parameters in CNS Gd-DTPA enhanced MR imaging. , 1991, Journal of computer assisted tomography.

[16]  Tammy Riklin-Raviv,et al.  De-noising of Contrast-Enhanced MRI Sequences by an Ensemble of Expert Deep Neural Networks , 2016, LABELS/DLMIA@MICCAI.

[17]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[18]  Ilan Shelef,et al.  Blood-brain barrier imaging in human neuropathologies. , 2014, Archives of medical research.

[19]  T W Redpath,et al.  The accuracy of pharmacokinetic parameter measurement in DCE-MRI of the breast at 3 T , 2010, Physics in medicine and biology.

[20]  V. Janardhan,et al.  Blood–brain barrier, reperfusion injury, and hemorrhagic transformation in acute ischemic stroke , 2012, Neurology.

[21]  Paul S. Tofts,et al.  Quantitative MRI of the brain : measuring changes caused by disease , 2003 .

[22]  Peyman Milanfar,et al.  Patch-Based Near-Optimal Image Denoising , 2012, IEEE Transactions on Image Processing.

[23]  Geoffrey E. Hinton Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[24]  Anne L. Martel A Fast Method of Generating Pharmacokinetic Maps from Dynamic Contrast-Enhanced Images of the Breast , 2006, MICCAI.

[25]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[26]  Stuart Crozier,et al.  Denoising of Dynamic Contrast-Enhanced MR Images Using Dynamic Nonlocal Means , 2010, IEEE Transactions on Medical Imaging.

[27]  M. Knopp,et al.  Estimating kinetic parameters from dynamic contrast‐enhanced t1‐weighted MRI of a diffusable tracer: Standardized quantities and symbols , 1999, Journal of magnetic resonance imaging : JMRI.

[28]  Joanna M. Wardlaw,et al.  Tracer kinetic modelling for DCE-MRI quantification of subtle blood–brain barrier permeability , 2016, NeuroImage.

[29]  Xiaokang Yang,et al.  Deep RNNs for video denoising , 2016, Optical Engineering + Applications.

[30]  David Mumford,et al.  Statistics of natural images and models , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[31]  C. Thng,et al.  Fundamentals of tracer kinetics for dynamic contrast‐enhanced MRI , 2011, Journal of magnetic resonance imaging : JMRI.

[32]  Alon Friedman,et al.  Overview and introduction: The blood–brain barrier in health and disease , 2012, Epilepsia.

[33]  Mannudeep K. Kalra,et al.  Low-Dose CT with a Residual Encoder-Decoder Convolutional Neural Network (RED-CNN) , 2017, ArXiv.

[34]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  David Rousseau,et al.  A 3-D spatio-temporal deconvolution approach for MR perfusion in the brain , 2014, Medical Image Anal..

[36]  Stephen M. Moore,et al.  The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository , 2013, Journal of Digital Imaging.

[37]  Michael Unser,et al.  Deep Convolutional Neural Network for Inverse Problems in Imaging , 2016, IEEE Transactions on Image Processing.

[38]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[39]  Guang-Zhong Yang,et al.  Bayesian Methods for Pharmacokinetic Models in Dynamic Contrast-Enhanced Magnetic Resonance Imaging , 2006, IEEE Transactions on Medical Imaging.

[40]  Héctor Pomares,et al.  Time Series Analysis and Forecasting , 2016 .

[41]  Stephen M Smith,et al.  Fast robust automated brain extraction , 2002, Human brain mapping.

[42]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[43]  John Scott Bridle,et al.  Probabilistic Interpretation of Feedforward Classification Network Outputs, with Relationships to Statistical Pattern Recognition , 1989, NATO Neurocomputing.

[44]  Maximilian Reiser,et al.  Quantification of cerebral blood flow, cerebral blood volume, and blood–brain‐barrier leakage with DCE‐MRI , 2009, Magnetic resonance in medicine.

[45]  Daniel Cremers,et al.  q-Space Deep Learning for Twelve-Fold Shorter and Model-Free Diffusion MRI Scans , 2015, MICCAI.

[46]  Feng Lin,et al.  Low-Dose CT With a Residual Encoder-Decoder Convolutional Neural Network , 2017, IEEE Transactions on Medical Imaging.

[47]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[48]  Tara N. Sainath,et al.  Improving deep neural networks for LVCSR using rectified linear units and dropout , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[49]  Josef Kittler,et al.  Pattern recognition : a statistical approach , 1982 .

[50]  Volker Schmid,et al.  Quantitative Analysis of Dynamic Contrast-Enhanced and Diffusion-Weighted Magnetic Resonance Imaging for Oncology in R , 2011 .

[51]  Svetlana Lublinsky,et al.  Analyzing the blood-brain barrier: the benefits of medical imaging in research and clinical practice. , 2015, Seminars in cell & developmental biology.

[52]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[53]  P. Tofts Modeling tracer kinetics in dynamic Gd‐DTPA MR imaging , 1997, Journal of magnetic resonance imaging : JMRI.

[54]  Ronald M. Summers,et al.  Gaussian Process Inference for Estimating Pharmacokinetic Parameters of Dynamic Contrast-Enhanced MR Images , 2012, MICCAI.

[55]  Ron Kimmel,et al.  Images as Embedded Maps and Minimal Surfaces: Movies, Color, Texture, and Volumetric Medical Images , 2000, International Journal of Computer Vision.

[56]  Stephen M. Smith,et al.  Improved Optimization for the Robust and Accurate Linear Registration and Motion Correction of Brain Images , 2002, NeuroImage.

[57]  P. Tofts,et al.  Measurement of the blood‐brain barrier permeability and leakage space using dynamic MR imaging. 1. Fundamental concepts , 1991, Magnetic resonance in medicine.

[58]  Joanna M. Wardlaw,et al.  Assessment of blood–brain barrier disruption using dynamic contrast-enhanced MRI. A systematic review , 2014, NeuroImage: Clinical.

[59]  Quoc V. Le,et al.  On optimization methods for deep learning , 2011, ICML.