Deep Learning-Guided Image Reconstruction from Incomplete Data

An approach to incorporate deep learning within an iterative image reconstruction framework to reconstruct images from severely incomplete measurement data is presented. Specifically, we utilize a convolutional neural network (CNN) as a quasi-projection operator within a least squares minimization procedure. The CNN is trained to encode high level information about the class of images being imaged; this information is utilized to mitigate artifacts in intermediate images produced by use of an iterative method. The structure of the method was inspired by the proximal gradient descent method, where the proximal operator is replaced by a deep CNN and the gradient descent step is generalized by use of a linear reconstruction operator. It is demonstrated that this approach improves image quality for several cases of limited-view image reconstruction and that using a CNN in an iterative method increases performance compared to conventional image reconstruction approaches. We test our method on several limited-view image reconstruction problems. Qualitative and quantitative results demonstrate state-of-the-art performance.

[1]  Melanie Grunwald,et al.  Foundations Of Image Science , 2016 .

[2]  Feng Lin,et al.  Low-Dose CT With a Residual Encoder-Decoder Convolutional Neural Network , 2017, IEEE Transactions on Medical Imaging.

[3]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[4]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[5]  Jean-Philippe Thiran,et al.  Learning the weight matrix for sparsity averaging in compressive imaging , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[6]  Yoshua Bengio,et al.  Plug & Play Generative Networks: Conditional Iterative Generation of Images in Latent Space , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Michael Möller,et al.  Learning Proximal Operators: Using Denoising Networks for Regularizing Inverse Imaging Problems , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[9]  Minh N. Do,et al.  Semantic Image Inpainting with Perceptual and Contextual Losses , 2016, ArXiv.

[10]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[11]  Hassan Mansour,et al.  Learning Optimal Nonlinearities for Iterative Thresholding Algorithms , 2015, IEEE Signal Processing Letters.

[12]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Andreas K. Maier,et al.  A Deep Learning Architecture for Limited-Angle Computed Tomography Reconstruction , 2017, Bildverarbeitung für die Medizin.

[14]  E. Sidky,et al.  Image reconstruction in circular cone-beam computed tomography by constrained, total-variation minimization , 2008, Physics in medicine and biology.

[15]  Leslie Ying,et al.  Accelerating magnetic resonance imaging via deep learning , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[16]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[17]  Jonathan Le Roux,et al.  Deep Unfolding: Model-Based Inspiration of Novel Deep Architectures , 2014, ArXiv.

[18]  Minh N. Do,et al.  Semantic Image Inpainting with Deep Generative Models , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Yu-Bin Yang,et al.  Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections , 2016, NIPS.

[20]  Mannudeep K. Kalra,et al.  Low-Dose CT with a Residual Encoder-Decoder Convolutional Neural Network (RED-CNN) , 2017, ArXiv.

[21]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[23]  Jian Sun,et al.  Deep ADMM-Net for Compressive Sensing MRI , 2016, NIPS.

[24]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[25]  Hiroshi Ishikawa,et al.  Globally and locally consistent image completion , 2017, ACM Trans. Graph..

[26]  Alexandros G. Dimakis,et al.  Compressed Sensing using Generative Models , 2017, ICML.

[27]  Ge Wang,et al.  A Perspective on Deep Imaging , 2016, IEEE Access.

[28]  Pavan K. Turaga,et al.  ReconNet: Non-Iterative Reconstruction of Images from Compressively Sensed Random Measurements , 2016, ArXiv.

[29]  Stephan Antholzer,et al.  Deep learning for photoacoustic tomography from sparse data , 2017, Inverse problems in science and engineering.

[30]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).