Model-based Deep Medical Imaging: the roadmap of generalizing iterative reconstruction model using deep learning

Medical imaging is playing a more and more important role in clinics. However, there are several issues in different imaging modalities such as slow imaging speed in MRI, radiation injury in CT and PET. Therefore, accelerating MRI, reducing radiation dose in CT and PET have been ongoing research topics since their invention. Usually, acquiring less data is a direct but important strategy to address these issues. However, less acquisition usually results in aliasing artifacts in reconstructions. Recently, deep learning (DL) has been introduced in medical image reconstruction and shown potential on significantly speeding up MR reconstruction and reducing radiation dose. In this paper, we propose a general framework on combining the reconstruction model with deep learning to maximize the potential of deep learning and model-based reconstruction, and give the examples to demonstrate the performance and requirements of unrolling different algorithms using deep learning.

[1]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[2]  Thomas Pock,et al.  Learning a variational network for reconstruction of accelerated MRI data , 2017, Magnetic resonance in medicine.

[3]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[4]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[5]  Xiang Zhu,et al.  Automatic Parameter Selection for Denoising Algorithms Using a No-Reference Measure of Image Content , 2010, IEEE Transactions on Image Processing.

[6]  Jian Sun,et al.  Deep ADMM-Net for Compressive Sensing MRI , 2016, NIPS.

[7]  Per Christian Hansen,et al.  Rank-Deficient and Discrete Ill-Posed Problems , 1996 .

[8]  Jacques Wainer,et al.  Automatic breast density classification using a convolutional neural network architecture search procedure , 2015, Medical Imaging.

[9]  Michael Elad,et al.  ESPIRiT—an eigenvalue approach to autocalibrating parallel MRI: Where SENSE meets GRAPPA , 2014, Magnetic resonance in medicine.

[10]  Morteza Mardani,et al.  Deep Generative Adversarial Neural Networks for Compressive Sensing MRI , 2019, IEEE Transactions on Medical Imaging.

[11]  Dong Liang,et al.  Adaptive Dictionary Learning in Sparse Gradient Domain for Image Recovery , 2013, IEEE Transactions on Image Processing.

[12]  Lei Liu,et al.  Automatic Convolutional Neural Architecture Search for Image Classification Under Different Scenes , 2019, IEEE Access.

[13]  Steen Moeller,et al.  Scan‐specific robust artificial‐neural‐networks for k‐space interpolation (RAKI) reconstruction: Database‐free deep learning for fast imaging , 2018, Magnetic resonance in medicine.

[14]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[15]  Bruce R. Rosen,et al.  Image reconstruction by domain-transform manifold learning , 2017, Nature.

[16]  Frank Hutter,et al.  Neural Architecture Search: A Survey , 2018, J. Mach. Learn. Res..

[17]  Dong Liang,et al.  DIMENSION: Dynamic MR imaging with both k‐space and spatial prior knowledge obtained via multi‐supervised network training , 2018, NMR in biomedicine.

[18]  Won-Ki Jeong,et al.  Compressed Sensing MRI Reconstruction Using a Generative Adversarial Network With a Cyclic Loss , 2017, IEEE Transactions on Medical Imaging.

[19]  Mathews Jacob,et al.  MoDL: Model-Based Deep Learning Architecture for Inverse Problems , 2017, IEEE Transactions on Medical Imaging.

[20]  L. Ying,et al.  Accelerating SENSE using compressed sensing , 2009, Magnetic resonance in medicine.

[21]  Daniel Rueckert,et al.  A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction , 2017, IEEE Transactions on Medical Imaging.

[22]  Guang Yang,et al.  DAGAN: Deep De-Aliasing Generative Adversarial Networks for Fast Compressed Sensing MRI Reconstruction , 2018, IEEE Transactions on Medical Imaging.

[23]  Daniel Rueckert,et al.  Convolutional Recurrent Neural Networks for Dynamic MR Image Reconstruction , 2017, IEEE Transactions on Medical Imaging.

[24]  Dong Liang,et al.  Improved parallel image reconstruction using feature refinement , 2018, Magnetic resonance in medicine.

[25]  Yujie Li,et al.  NAS-Unet: Neural Architecture Search for Medical Image Segmentation , 2019, IEEE Access.

[26]  Jong Chul Ye,et al.  Deep learning with domain adaptation for accelerated projection‐reconstruction MR , 2018, Magnetic resonance in medicine.

[27]  Dong Liang,et al.  Iterative feature refinement for accurate undersampled MR image reconstruction , 2016, Physics in medicine and biology.

[28]  Yoram Bresler,et al.  MR Image Reconstruction From Highly Undersampled k-Space Data by Dictionary Learning , 2011, IEEE Transactions on Medical Imaging.

[29]  D. O. Walsh,et al.  Adaptive reconstruction of phased array MR imagery , 2000, Magnetic resonance in medicine.

[30]  Zongben Xu,et al.  ADMM-CSNet: A Deep Learning Approach for Image Compressive Sensing , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Justin P. Haldar,et al.  The Fourier radial error spectrum plot: A more nuanced quantitative evaluation of image reconstruction quality , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[32]  Leslie Ying,et al.  Accelerating magnetic resonance imaging via deep learning , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[33]  S. T. Nichols,et al.  Quantitative evaluation of several partial fourier reconstruction algorithms used in mri , 1993, Magnetic resonance in medicine.

[34]  Taeseong Kim,et al.  KIKI‐net: cross‐domain convolutional neural networks for reconstructing undersampled magnetic resonance images , 2018, Magnetic resonance in medicine.