A Novel Registration & Colorization Technique for Thermal to Cross Domain Colorized Images

Thermal images can be obtained as either grayscale images or pseudo colored images based on the thermal profile of the object being captured. We present a novel registration method that works on images captured via multiple thermal imagers irrespective of make and internal resolution as well as a colorization scheme that can be used to obtain a colorized thermal image which is similar to an optical image, while retaining the information of the thermal profile as a part of the output, thus providing information of both domains jointly. We call this a cross domain colorized image. We also outline a new public thermal-optical paired database that we are presenting as a part of this paper, containing unique data points obtained via multiple thermal imagers. Finally, we compare the results with prior literature, show how our results are different and discuss on some future work that can be explored further in this domain as well.

[1]  Patricia L. Suárez,et al.  Learning to Colorize Infrared Images , 2017, PAAMS.

[2]  Angel Domingo Sappa,et al.  Infrared Image Colorization Based on a Triplet DCGAN Architecture , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[3]  Christoph H. Lampert,et al.  Probabilistic Image Colorization , 2017, BMVC.

[4]  Aditya Deshpande,et al.  Learning Diverse Image Colorization , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Michael Felsberg,et al.  Generating Visible Spectrum Images from Thermal Infrared , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[6]  Hendrik P. A. Lensch,et al.  Infrared Colorization Using Deep Convolutional Neural Networks , 2016, 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA).

[7]  Harry Shum,et al.  Natural Image Colorization , 2007, Rendering Techniques.

[8]  Dani Lischinski,et al.  Colorization using optimization , 2004, ACM Trans. Graph..

[9]  Namil Kim,et al.  Multispectral pedestrian detection: Benchmark dataset and baseline , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Peng Xu,et al.  Multi-Spectral Stereo Image Matching Based On Adaptive Window , 2018 .

[11]  Jun Tang,et al.  A color image segmentation algorithm based on region growing , 2010, 2010 2nd International Conference on Computer Engineering and Technology.

[12]  Nenghai Yu,et al.  Stereoscopic Neural Style Transfer , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[13]  Gregory Shakhnarovich,et al.  Learning Representations for Automatic Colorization , 2016, ECCV.

[14]  Jun-Cheng Chen,et al.  An adaptive edge detection based colorization algorithm and its applications , 2005, ACM Multimedia.

[15]  Vijay John,et al.  Deep Learning Thermal Image Translation for Night Vision Perception , 2020, ACM Trans. Intell. Syst. Technol..

[16]  Nenghai Yu,et al.  Coherent Online Video Style Transfer , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[17]  Léon Bottou,et al.  Wasserstein GAN , 2017, ArXiv.

[18]  Quan Pan,et al.  Color based grayscale-fused image enhancement algorithm for video surveillance , 2004, Third International Conference on Image and Graphics (ICIG'04).

[19]  Edgar Simo-Serra,et al.  Let there be Color!: Joint End-to-end Learning of Global and Local Image Priors for Automatic Image Colorization with Simultaneous Classification , 2016 .

[20]  Nenghai Yu,et al.  StyleBank: An Explicit Representation for Neural Image Style Transfer , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  David Pfau,et al.  Unrolled Generative Adversarial Networks , 2016, ICLR.

[22]  Junsheng Shi,et al.  Intelligent Colorization for Thermal Infrared Image Based on CNN , 2020, 2020 IEEE International Conference on Information Technology,Big Data and Artificial Intelligence (ICIBA).

[23]  Yong Yu,et al.  Unsupervised Diverse Colorization via Generative Adversarial Networks , 2017, ECML/PKDD.

[24]  Deepu Rajan,et al.  Image colorization using similar images , 2012, ACM Multimedia.

[25]  Hua Huang,et al.  No-reference image quality assessment based on spatial and spectral entropies , 2014, Signal Process. Image Commun..

[26]  Qian Chen,et al.  Thermal Infrared Colorization via Conditional Generative Adversarial Network , 2018, Infrared Physics & Technology.

[27]  Klaus Mueller,et al.  Transferring color to greyscale images , 2002, ACM Trans. Graph..

[28]  Wojciech Zaremba,et al.  Improved Techniques for Training GANs , 2016, NIPS.

[29]  Bin Sheng,et al.  Deep Colorization , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).