Turnip: Time-Series U-Net With Recurrence For NIR Imaging PPG

Near-Infrared (NIR) videos of faces acquired with active illumination for the problem of estimating the photoplethysmogram (PPG) signal from a distance have demonstrated improved robustness to ambient illumination. Contrary to the multichannel RGB-based solutions, prior work in the NIR regime has been purely model-based and has exploited sparsity of the PPG signal in the frequency domain. In contrast, we propose in this paper a modular neural network-based framework for estimating the remote PPG (rPPG) signal. We test our approach on two challenging datasets where the subjects are inside a car and can have a lot of head motion. We show that our method outperforms existing model-based methods as well as end-to-end deep learning methods for rPPG estimation from NIR videos. IEEE International Conference on Image Processing (ICIP) 2021 c © 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Mitsubishi Electric Research Laboratories, Inc. 201 Broadway, Cambridge, Massachusetts 02139 TURNIP: TIME-SERIES U-NET WITH RECURRENCE FOR NIR IMAGING PPG Armand Comas∗2 Tim K. Marks Hassan Mansour Suhas Lohit Yechi Ma∗3 Xiaoming Liu Mitsubishi Electric Research Laboratories (MERL), Cambridge, MA, USA Northeastern University, Boston, MA, USA Princeton University, Princeton, NJ, USA Michigan State University, East Lansing, MI, USA