Adaptive Fusion of RGB/NIR Signals Based on Face/Background Cross-Spectral Analysis for Heart Rate Estimation

We propose a method for heart rate (HR) estimation that is robust to various situations such as bright, low-light, and varying illumination scenes. We capture temporal variations in the pixel values owing to person’s cardiac pulse by using a camera that records red, green, and blue (RGB) and near-infrared (NIR) information. The key novelty of our method is to introduce a scheme for adaptive fusion of RGB and NIR signals for HR estimation, by analyzing variations in the background illuminations. RGB signals will be a good cue for HR estimation under bright scenes. In contrast, NIR signals are more reliable in HR estimation than RGB ones in complex illumination scenes, because NIR signals can be captured independent to changes in the background illuminations. By measuring correlations of signals between background and face regions, we adaptively utilize RGB and NIR signals for HR estimation. Experiments demonstrate the effectiveness of our method.