Multiscale top-hat selection transform based infrared and visual image fusion with emphasis on extracting regions of interest

Abstract To effectively combine regions of interest in original infrared and visual images, an adaptively weighted infrared and visual image fusion algorithm is developed based on the multiscale top-hat selection transform. First, the multiscale top-hat selection transform using multiscale structuring elements with increasing sizes is discussed. Second, the image regions of the original infrared and visual images at each scale are extracted by using the multiscale top-hat selection transform. Third, the final fusion regions are constructed from the extracted multiscale image regions. Finally, the final fusion regions are combined into a base image calculated from the original images to form the final fusion result. The combination of the final fusion regions uses the adaptive weight strategy, and the weights are adaptively obtained based on the importance of the extracted features. In the paper, we compare seven image fusion methods: wavelet pyramid algorithm (WP), shift invariant discrete wavelet transform algorithm (SIDWT), Laplacian pyramid algorithm (LP), morphological pyramid algorithm (MP), multiscale morphology based algorithm (MSM), center-surround top-hat transform based algorithm (CSTHT), and the proposed multiscale top-hat selection transform based algorithm. These seven methods are compared over five different publicly available image sets using three metrics of spatial frequency, mean gradient, and Q. The results show that the proposed algorithm is effective and may be useful for the applications related to the infrared and visual image fusion.