Qualitative evaluations and comparisons of six night-vision colorization methods

Current multispectral night vision (NV) colorization techniques can manipulate images to produce colorized images that closely resemble natural scenes. The colorized NV images can enhance human perception by improving observer object classification and reaction times especially for low light conditions. This paper focuses on the qualitative (subjective) evaluations and comparisons of six NV colorization methods. The multispectral images include visible (Red-Green- Blue), near infrared (NIR), and long wave infrared (LWIR) images. The six colorization methods are channel-based color fusion (CBCF), statistic matching (SM), histogram matching (HM), joint-histogram matching (JHM), statistic matching then joint-histogram matching (SM-JHM), and the lookup table (LUT). Four categries of quality measurements are used for the qualitative evaluations, which are contrast, detail, colorfulness, and overall quality. The score of each measurement is rated from 1 to 3 scale to represent low, average, and high quality, respectively. Specifically, high contrast (of rated score 3) means an adequate level of brightness and contrast. The high detail represents high clarity of detailed contents while maintaining low artifacts. The high colorfulness preserves more natural colors (i.e., closely resembles the daylight image). Overall quality is determined from the NV image compared to the reference image. Nine sets of multispectral NV images were used in our experiments. For each set, the six colorized NV images (produced from NIR and LWIR images) are concurrently presented to users along with the reference color (RGB) image (taken at daytime). A total of 67 subjects passed a screening test (“Ishihara Color Blindness Test”) and were asked to evaluate the 9-set colorized images. The experimental results showed the quality order of colorization methods from the best to the worst: CBCF < SM < SM-JHM < LUT < JHM < HM. It is anticipated that this work will provide a benchmark for NV colorization and for quantitative evaluation using an objective metric such as objective evaluation index (OEI).

[1]  David M. Craig,et al.  Progress on color night vision: visible/IR fusion, perception and search, and low-light CCD imaging , 1996, Defense, Security, and Sensing.

[2]  Alexander Toet,et al.  Natural colour mapping for multiband nightvision imagery , 2003, Inf. Fusion.

[3]  Benkang Chang,et al.  Objective quality evaluation of visible and infrared color fusion image , 2011 .

[4]  D. Ruderman,et al.  Statistics of cone responses to natural images: implications for visual coding , 1998 .

[5]  D. Burr,et al.  Mach bands are phase dependent , 1986, Nature.

[6]  Alexander Toet,et al.  Perceptual evaluation of different image fusion schemes , 2001, SPIE Defense + Commercial Sensing.

[7]  D. Hill,et al.  Medical image registration , 2001, Physics in medicine and biology.

[8]  Yufeng Zheng,et al.  An overview of night vision colorization techniques using multispectral images: From color fusion to color mapping , 2012, 2012 International Conference on Audio, Language and Image Processing.

[9]  Daniel Malacara,et al.  Color Vision and Colorimetry: Theory and Applications , 2002 .

[10]  Jason S. McCarley,et al.  Perceptual Ability with Real-World Nighttime Scenes: Image-Intensified, Infrared, and Fused-Color Imagery , 1999, Hum. Factors.

[11]  Michel Defrise,et al.  Symmetric Phase-Only Matched Filtering of Fourier-Mellin Transforms for Image Registration and Recognition , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Yufeng Zheng,et al.  Advanced discrete wavelet transform fusion algorithm and its optimization by using the metric of image quality index , 2005 .

[13]  John T. Vargo.  Evaluation of Operator Performance Using True Color and Artificial Color in Natural Scene Perception. , 1999 .

[14]  Yufeng Zheng,et al.  A channel-based color fusion technique using multispectral images for night vision enhancement , 2011, Optical Engineering + Applications.

[15]  Zheng Liu,et al.  Objective Assessment of Multiresolution Image Fusion Algorithms for Context Enhancement in Night Vision: A Comparative Study , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Jason S. McCarley,et al.  Human Perception of Sensor-Fused Imagery , 2001, Interpreting Remote Sensing Imagery.

[17]  Yufeng Zheng,et al.  Qualitative and quantitative comparisons of multispectral night vision colorization techniques , 2012, Optical Engineering.

[18]  Yufeng Zheng,et al.  A local-coloring method for night-vision colorization utilizing image analysis and fusion , 2008, Inf. Fusion.

[19]  Yufeng Zheng,et al.  An objective evaluation metric for color image fusion , 2012, Defense + Commercial Sensing.

[20]  Michael H. Brill,et al.  Color appearance models , 1998 .

[21]  Alexander Toet,et al.  Method for applying daytime colors to nighttime imagery in realtime , 2008, SPIE Defense + Commercial Sensing.

[22]  Alexander Toet,et al.  Fusion of visible and thermal imagery improves situational awareness , 1997 .

[23]  Miao Ma,et al.  New method to quality evaluation for image fusion using gray relational analysis , 2005 .