Multi-Modal Deep Guided Filtering for Comprehensible Medical Image Processing

Deep learning-based image processing is capable of creating highly appealing results. However, it is still widely considered as a “blackbox” transformation. In medical imaging, this lack of comprehensibility of the results is a sensitive issue. The integration of known operators into the deep learning environment has proven to be advantageous for the comprehensibility and reliability of the computations. Consequently, we propose the use of the locally linear guided filter in combination with a learned guidance map for general purpose medical image processing. The output images are only processed by the guided filter while the guidance map can be trained to be task-optimal in an end-to-end fashion. We investigate the performance based on two popular tasks: image super resolution and denoising. The evaluation is conducted based on pairs of multi-modal magnetic resonance imaging and cross-modal computed tomography and magnetic resonance imaging datasets. For both tasks, the proposed approach is on par with state-of-the-art approaches. Additionally, we can show that the input image’s content is almost unchanged after the processing which is not the case for conventional deep learning approaches. On top, the proposed pipeline offers increased robustness against degraded input as well as adversarial attacks.

[1]  Andreas K. Maier,et al.  Multi-Modal Super-Resolution with Deep Guided Filtering , 2019, Bildverarbeitung für die Medizin.

[2]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[3]  Andreas K. Maier,et al.  MR to X-Ray Projection Image Synthesis , 2017, ArXiv.

[4]  Lovedeep Gondara,et al.  Medical Image Denoising Using Convolutional Denoising Autoencoders , 2016, 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW).

[5]  Ning Xu,et al.  Wide Activation for Efficient and Accurate Image Super-Resolution , 2018, ArXiv.

[6]  Jian Sun,et al.  Guided Image Filtering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Percy Liang,et al.  Understanding Black-box Predictions via Influence Functions , 2017, ICML.

[8]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[9]  Ge Wang,et al.  Vision 20/20: Simultaneous CT-MRI--Next chapter of multimodality imaging. , 2015, Medical physics.

[10]  Francesco Renna,et al.  On instabilities of deep learning in image reconstruction and the potential costs of AI , 2019, Proceedings of the National Academy of Sciences.

[11]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[12]  Kaiqi Huang,et al.  Fast End-to-End Trainable Guided Filter , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[13]  Christian Riess,et al.  CONRAD--a software framework for cone-beam imaging in radiology. , 2013, Medical physics.

[14]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[15]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[16]  Jonas Adler,et al.  Learned Primal-Dual Reconstruction , 2017, IEEE Transactions on Medical Imaging.

[17]  Konstantinos Kamnitsas,et al.  Multi-input Cardiac Image Super-Resolution Using Convolutional Neural Networks , 2016, MICCAI.

[18]  Zhengguo Li,et al.  Gradient Domain Guided Image Filtering , 2015, IEEE Transactions on Image Processing.

[19]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[20]  Chi-Hieu Pham,et al.  Brain MRI super-resolution using deep 3D convolutional networks , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[21]  Andrea Vedaldi,et al.  Instance Normalization: The Missing Ingredient for Fast Stylization , 2016, ArXiv.

[22]  Rebecca Fahrig,et al.  MR-projection imaging with perspective distortion as in X-ray fluoroscopy for interventional X / MR-hybrid applications , 2018 .

[23]  Michel Desvignes,et al.  Spatio-Temporal Multiscale Denoising of Fluoroscopic Sequence , 2016, IEEE Transactions on Medical Imaging.

[24]  Charles B. Fleming,et al.  Opening the Black Box: Using Process Evaluation Measures to Assess Implementation and Theory Building , 1999, American journal of community psychology.

[25]  Thomas Pock,et al.  Learning a variational network for reconstruction of accelerated MRI data , 2017, Magnetic resonance in medicine.

[26]  Tobias Würfl,et al.  Deriving Neural Network Architectures using Precision Learning: Parallel-to-fan beam Conversion , 2018, GCPR.

[27]  Pan He,et al.  Adversarial Examples: Attacks and Defenses for Deep Learning , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[28]  N J Pelc,et al.  A truly hybrid interventional MR/X‐ray system: Feasibility demonstration , 2001, Journal of magnetic resonance imaging : JMRI.

[29]  David A. Wagner,et al.  Towards Evaluating the Robustness of Neural Networks , 2016, 2017 IEEE Symposium on Security and Privacy (SP).

[30]  Ron Kikinis,et al.  3D Slicer , 2012, 2004 2nd IEEE International Symposium on Biomedical Imaging: Nano to Macro (IEEE Cat No. 04EX821).

[31]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Naftali Tishby,et al.  Opening the Black Box of Deep Neural Networks via Information , 2017, ArXiv.

[33]  Shutao Li,et al.  Image Fusion With Guided Filtering , 2013, IEEE Transactions on Image Processing.

[34]  Andreas K. Maier,et al.  Learning with Known Operators reduces Maximum Training Error Bounds , 2019, Nature Machine Intelligence.

[35]  Andreas K. Maier,et al.  Deep Learning Computed Tomography , 2016, MICCAI.

[36]  Nassir Navab,et al.  Simulation of Realistic Low Dose Fluoroscopic Images from their High Dose Counterparts , 2018, Bildverarbeitung für die Medizin.

[37]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[38]  Thomas S. Huang,et al.  Wide-activated Deep Residual Networks based Restoration for BPG-compressed Images , 2018, CVPR Workshops.

[39]  T. Redpath.  Signal-to-noise ratio in MRI. , 1998, The British journal of radiology.