Image Reconstruction by Splitting Deep Learning Regularization from Iterative Inversion

Image reconstruction from downsampled and corrupted measurements, such as fast MRI and low dose CT, is mathematically ill-posed inverse problem. In this work, we propose a general and easy-to-use reconstruction method based on deep learning techniques. In order to address the intractable inversion of general inverse problems, we propose to train a network to refine intermediate images from classical reconstruction procedure to the ground truth, i.e. the intermediate images that satisfy the data consistence will be fed into some chosen denoising networks or generative networks for denoising and removing artifact in each iterative stage. The proposed approach involves only techniques of conventional image reconstruction and usual image representation/denoising deep network learning, without a specifically designed and complicated network structures for a certain physical forward operator. Extensive experiments on MRI reconstruction applied with both stack auto-encoder networks and generative adversarial nets demonstrate the efficiency and accuracy of the proposed method compared with other image reconstruction algorithms.

[1]  Daniel Rueckert,et al.  A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction , 2017, IEEE Transactions on Medical Imaging.

[2]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Song Han,et al.  Deep Generative Adversarial Networks for Compressed Sensing Automates MRI , 2017, ArXiv.

[4]  S. Osher,et al.  Image restoration: Total variation, wavelet frames, and beyond , 2012 .

[5]  Enhong Chen,et al.  Image Denoising and Inpainting with Deep Neural Networks , 2012, NIPS.

[6]  M. Bernstein,et al.  Effect of windowing and zero‐filled reconstruction of MRI data on spatial resolution and acquisition strategy , 2001, Journal of magnetic resonance imaging : JMRI.

[7]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[8]  Katherine A. Heller,et al.  InverseNet: Solving Inverse Problems with Splitting Networks , 2017, ArXiv.

[9]  Jonas Adler,et al.  Solving ill-posed inverse problems using iterative deep neural networks , 2017, ArXiv.

[10]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[11]  Jürgen Schmidhuber,et al.  Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction , 2011, ICANN.

[12]  Jian Sun,et al.  Deep ADMM-Net for Compressive Sensing MRI , 2016, NIPS.

[13]  M. Fortin,et al.  Augmented Lagrangian methods : applications to the numerical solution of boundary-value problems , 1983 .

[14]  Jonas Adler,et al.  Learned Primal-Dual Reconstruction , 2017, IEEE Transactions on Medical Imaging.

[15]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[16]  Thomas Pock,et al.  Learning a variational network for reconstruction of accelerated MRI data , 2017, Magnetic resonance in medicine.

[17]  Dacheng Tao,et al.  Non-Local Auto-Encoder With Collaborative Stabilization for Image Restoration , 2016, IEEE Transactions on Image Processing.

[18]  Thomas Hofmann,et al.  Greedy Layer-Wise Training of Deep Networks , 2007 .