Dual-mode imaging system for non-contact heart rate estimation during night

To estimate heart rate (HR) during night remotely and unobtrusively, we use a far-infrared camera and an RGB-Infrared camera to develop a dual-mode imaging system. The RGB or infrared images are first registered by the far-infrared images through the affine transformation. A custom-made cascade face classifier, which contains the conventional Adaboost model and fully convolutional network, was applied for the detection of the face in registered infrared images. The fully convolutional network was trained by 32K images from the PASCAL dataset. Subsequently, two facial tissues viz., mouth and nose were determined by the discriminative regression via the coordinate conversions of three selected landmarks. The spatio-temporal context learning was utilized to track the mouth and nose regions in the far-infrared image sequence. Then, the raw image feature was extracted from these two regions of interest. Finally, a state-of-art signal analysis method was explored to calculate HR in the time domain of the extracted raw signal. With respect to the validation experiment, we established the dual-mode sleep video database to verify the performances of the proposed system and algorithms. All videos in database were filmed under the environments where actual illumination intensity ranged from 0 to 3 Lux. The obtained results demonstrated that the determination coefficient (R2) was 0.933 for HR estimation in linear regression analysis. The Bland-Altman analysis showed that almost all the data points located within the 95% upper and lower limits of agreement, which were 4.293 and −5.293 bpm, respectively. Therefore, the proposed technique is efficient for the non-contact and unobtrusive HR estimation during night.