Adaptive Medical Image Deep Color Perception Algorithm

The existing medical imaging technologies have little consideration on color information, thus most of medical images are gray. Classical hand-craft features-based methods have obtained unsatisfactory results in colorizing medical images. Moreover, these methods ignore the deep feature of medical images that represent pathology and color information. In this paper, we propose a novel method that iteratively colorizes grayscale medical images under preserving content in fine-tuned deep neural network. To the best of our knowledge, there is no currently work that attempts to colorize the medical image by using deep neural network. Specifically, we propose Y-loss which is defined as nonlinear combination of $\ell _{1}$ and $\ell _{2}$ norm to preserve content invariance between target and colorized medical image. Then, adaptive reference image search algorithm is introduced to code reference and target medical image with D-hash and search reference image in hash code automatically, which free the manual selection of the reference image. Extensive experiment results show that the proposed method can generate higher quality colored medical image than recent state-of-the-art methods, and can be approved by the doctor. The objective evaluation (PSNR and SSIM) outperform an average increment 24% and 47% than baseline method, respectively. Our code is available at: https://github.com/Tongshiyue/Adaptive-medical-image-deep-color-perception-algorithm.

[1]  Jorge Nocedal,et al.  Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization , 1997, TOMS.

[2]  Meng Zhou,et al.  Color perception algorithm of medical images using density peak based hierarchical clustering , 2019, Biomed. Signal Process. Control..

[3]  Dinggang Shen,et al.  Scalable High-Performance Image Registration Framework by Unsupervised Deep Feature Representations Learning , 2016, IEEE Transactions on Biomedical Engineering.

[4]  Jie Li,et al.  Unsupervised Semantic-Preserving Adversarial Hashing for Image Search , 2019, IEEE Transactions on Image Processing.

[5]  Khan A. Wahid,et al.  Efficient Color Reproduction Algorithm for Endoscopic Images Based on Dynamic Color Map , 2016 .

[6]  Chuan Li,et al.  Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Sylvain Paris,et al.  Deep Photo Style Transfer , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Jieqing Tan,et al.  Color Transfer Based on Combining Subtractive Clustering with FCM Clustering , 2007, 2007 10th IEEE International Conference on Computer-Aided Design and Computer Graphics.

[9]  Lizhuang Ma,et al.  Color transfer in correlated color space , 2006, VRCIA '06.

[10]  Klaus Mueller,et al.  Transferring color to greyscale images , 2002, ACM Trans. Graph..

[11]  Marcin Woźniak,et al.  An Adaptive Local Descriptor Embedding Zernike Moments for Image Matching , 2019, IEEE Access.

[12]  Oliver Y. Chén,et al.  Joint Classification and Prediction CNN Framework for Automatic Sleep Stage Classification , 2018, IEEE Transactions on Biomedical Engineering.

[13]  Ling Shao,et al.  Robust Visual Tracking Based on Improved Perceptual Hashing for Robot Vision , 2015, ICIRA.

[14]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.

[16]  Russell Zaretzki,et al.  Emotion transfer for images based on color combinations , 2013, ArXiv.

[17]  Jiaming Guo,et al.  An edge-refined vectorized deep colorization model for grayscale-to-color images , 2018, Neurocomputing.

[18]  Hiroshi Ishikawa,et al.  Let there be color! , 2016, ACM Trans. Graph..

[19]  Paul Sanzenbacher,et al.  Automated Deep Photo Style Transfer , 2019, ArXiv.

[20]  Andrea Vedaldi,et al.  Understanding deep image representations by inverting them , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Jia Liu,et al.  A Cascaded Deep Convolutional Neural Network for Joint Segmentation and Genotype Prediction of Brainstem Gliomas , 2018, IEEE Transactions on Biomedical Engineering.

[22]  Chao Li,et al.  Shared Predictive Cross-Modal Deep Quantization , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[23]  Bin Zhou,et al.  A regional adaptive variational PDE model for computed tomography image reconstruction , 2019, Pattern Recognit..

[24]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[25]  Hod Lipson,et al.  Understanding Neural Networks Through Deep Visualization , 2015, ArXiv.

[26]  Ting Zhao,et al.  Pyramid Feature Attention Network for Saliency Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Lipo Wang,et al.  Deep Learning Applications in Medical Image Analysis , 2018, IEEE Access.

[28]  Xianhua Zeng,et al.  Color Perception Algorithm of Medical Images Using Multi-features Fusion , 2018 .

[29]  Noura A. Semary,et al.  A Proposed HSV-Based Pseudo Coloring Scheme for Enhancing Medical Image , 2018 .

[30]  Leon A. Gatys,et al.  Texture Synthesis Using Convolutional Neural Networks , 2015, NIPS.

[31]  Ning Chunyu Medical Image Pseudo-color Processing Based on Object Enhancement , 2012 .

[32]  Muhammad Usman Ghani Khan,et al.  Medical Image Colorization for Better Visualization and Segmentation , 2017, Annual Conference on Medical Image Understanding and Analysis.