Accelerating Super-Resolution and Visual Task Analysis in Medical Images

Medical images are acquired at different resolutions based on clinical goals or available technology. In general, however, high-resolution images with fine structural details are preferred for visual task analysis. Recognizing this significance, several deep learning networks have been proposed to enhance medical images for reliable automated interpretation. These deep networks are often computationally complex and require a massive number of parameters, which restrict them to highly capable computing platforms with large memory banks. In this paper, we propose an efficient deep learning approach, called Hydra, which simultaneously reduces computational complexity and improves performance. The Hydra consists of a trunk and several computing heads. The trunk is a super-resolution model that learns the mapping from low-resolution to high-resolution images. It has a simple architecture that is trained using multiple scales at once to minimize a proposed learning-loss function. We also propose to append multiple task-specific heads to the trained Hydra trunk for simultaneous learning of multiple visual tasks in medical images. The Hydra is evaluated on publicly available chest X-ray image collections to perform image enhancement, lung segmentation, and abnormality classification. Our experimental results support our claims and demonstrate that the proposed approach can improve the performance of super-resolution and visual task analysis in medical images at a remarkably reduced computational cost.

[1]  Andre Mouton,et al.  On the relevance of denoising and artefact reduction in 3D segmentation and classification within complex computed tomography imagery. , 2019, Journal of X-ray science and technology.

[2]  Farida Cheriet,et al.  A multi-scale tensor voting approach for small retinal vessel segmentation in high resolution fundus images , 2016, Comput. Medical Imaging Graph..

[3]  Chang-Woo Ryu,et al.  Comparison of Low- and Standard-Dose CT for the Diagnosis of Acute Appendicitis: A Meta-Analysis. , 2017, AJR. American journal of roentgenology.

[4]  Radu Tudor Ionescu,et al.  Convolutional Neural Networks With Intermediate Loss for 3D Super-Resolution of CT and MRI Scans , 2020, IEEE Access.

[5]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[6]  Vasudevan Lakshminarayanan,et al.  Application of an enhanced deep super-resolution network in retinal image analysis , 2020, BiOS.

[7]  S. Tashiro,et al.  Estimation of the effects of medical diagnostic radiation exposure based on DNA damage , 2018, Journal of radiation research.

[8]  Andrey S. Krylov,et al.  Image Interpolation by Super-Resolution , 2006 .

[9]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[10]  Neeraj Sharma,et al.  Segmentation and classification of medical images using texture-primitive features: Application of BAM-type artificial neural network , 2008, Journal of medical physics.

[11]  Clement J. McDonald,et al.  Lung Segmentation in Chest Radiographs Using Anatomical Atlases With Nonrigid Registration , 2014, IEEE Transactions on Medical Imaging.

[12]  Carol C Wu,et al.  Augmenting the National Institutes of Health Chest Radiograph Dataset with Expert Annotations of Possible Pneumonia. , 2019, Radiology. Artificial intelligence.

[13]  Lin Zheng,et al.  Fast low-dose Computed Tomography image Super-Resolution Reconstruction via Sparse coding and Random Forests , 2019, 2019 IEEE 8th Joint International Information Technology and Artificial Intelligence Conference (ITAIC).

[14]  Changhee Han,et al.  Synthesizing Diverse Lung Nodules Wherever Massively: 3D Multi-Conditional GAN-Based CT Image Augmentation for Object Detection , 2019, 2019 International Conference on 3D Vision (3DV).

[15]  Paramartha Dutta,et al.  Evolution of Image Segmentation using Deep Convolutional Neural Network: A Survey , 2020, Knowl. Based Syst..

[16]  Xiaoou Tang,et al.  Accelerating the Super-Resolution Convolutional Neural Network , 2016, ECCV.

[17]  Wei Wang,et al.  Deep Learning for Single Image Super-Resolution: A Brief Review , 2018, IEEE Transactions on Multimedia.

[18]  Samet Aymaz,et al.  Multi-focus image fusion for different datasets with super-resolution using gradient-based new fusion rule , 2020, Multimedia Tools and Applications.

[19]  Dinggang Shen,et al.  Machine Learning in Medical Imaging , 2012, Lecture Notes in Computer Science.

[20]  Russell C. Hardie,et al.  A Computationally Efficient U-Net Architecture for Lung Segmentation in Chest Radiographs , 2019, 2019 IEEE National Aerospace and Electronics Conference (NAECON).

[21]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Juan Yu,et al.  Multi-resolution convolutional networks for chest X-ray radiograph based lung nodule detection , 2019, Artif. Intell. Medicine.

[23]  Weisheng Li,et al.  Low-dose chest X-ray image super-resolution using generative adversarial nets with spectral normalization , 2020, Biomed. Signal Process. Control..

[24]  Giancarlo Mauri,et al.  A novel framework for MR image segmentation and quantification by using MedGA , 2019, Comput. Methods Programs Biomed..

[25]  Horst Bischof,et al.  Fast and accurate image upscaling with super-resolution forests , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Zhou Wang,et al.  On the Mathematical Properties of the Structural Similarity Index , 2012, IEEE Transactions on Image Processing.

[27]  Kin-Man Lam,et al.  Image Super-resolution via Feature-augmented Random Forest , 2017, Signal Process. Image Commun..

[28]  Steven C. H. Hoi,et al.  Deep Learning for Image Super-Resolution: A Survey , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Dwarikanath Mahapatra,et al.  Progressive Generative Adversarial Networks for Medical Image Super resolution , 2019, ArXiv.

[30]  Guang Li,et al.  CT Super-Resolution GAN Constrained by the Identical, Residual, and Cycle Learning Ensemble (GAN-CIRCLE) , 2018, IEEE Transactions on Medical Imaging.

[31]  R. Keys Cubic convolution interpolation for digital image processing , 1981 .

[32]  Xingying Li,et al.  Regularized super-resolution restoration algorithm for single medical image based on fuzzy similarity fusion , 2019, EURASIP J. Image Video Process..

[33]  Giancarlo Mauri,et al.  MedGA: A novel evolutionary method for image enhancement in medical imaging systems , 2019, Expert Syst. Appl..

[34]  Hui Fan,et al.  Multifocus Image Fusion Using Wavelet-Domain-Based Deep CNN , 2019, Comput. Intell. Neurosci..

[35]  Zhou Wang,et al.  Multiscale structural similarity for image quality assessment , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[36]  Kohei Arai Advances in Computer Vision - Proceedings of the 2019 Computer Vision Conference, CVC 2019, Las Vegas, Nevada, USA, 25-26 April 2019, Volume 1 , 2020, CVC.

[37]  P. Salminen,et al.  The Accuracy of Low-dose Computed Tomography Protocol in Patients With Suspected Acute Appendicitis: The OPTICAP Study. , 2020, Annals of surgery.

[38]  Stefan Jaeger,et al.  Two public chest X-ray datasets for computer-aided screening of pulmonary diseases. , 2014, Quantitative imaging in medicine and surgery.

[39]  Nicolas Papadakis,et al.  GraphX$^{NET}-$ Chest X-Ray Classification Under Extreme Minimal Supervision , 2019, 1907.10085.

[40]  K. Doi,et al.  Development of a digital image database for chest radiographs with and without a lung nodule: receiver operating characteristic analysis of radiologists' detection of pulmonary nodules. , 2000, AJR. American journal of roentgenology.