Good match exploration for thermal infrared face recognition based on YWF-SIFT with multi-scale fusion

Abstract Stable local feature detection is a critical prerequisite in the problem of infrared (IR) face recognition. Recently, Scale Invariant Feature Transform (SIFT) is introduced for feature detection in an infrared face frame, which is achieved by applying a simple and effective averaging window with SIFT termed as Y-styled Window Filter (YWF). However, the thermal IR face frame has an intrinsic characteristic such as lack of feature points (keypoints); therefore, the performance of the YWF-SIFT method will be inevitably influenced when it was used for IR face recognition. In this paper, we propose a novel method combining multi-scale fusion with YWF-SIFT to explore more good feature matches. The multi-scale fusion is performed on a thermal IR frame and a corresponding auxiliary visual frame generated from an off-the-shelf low-cost visual camera. The fused image is more informative, and typically contains much more stable features. Besides, the use of YWF-SIFT method enables us to establish feature correspondences more accurately. Quantitative experimental results demonstrate that our algorithm is able to significantly improve the quantity of feature points by approximately 38%. As a result, the performance of YWF-SIFT with multi-scale fusion is enhanced about 12% in infrared human face recognition.