Physiological Signal Preserving Video Compression for Remote Photoplethysmography

The consumer-level digital camera has become a physiological signal monitoring sensor due to the rapid growth of the remote photoplethysmography (rPPG) technique. However, rPPG suffers from the artifacts caused by video compression, a technique that widely exists in nearly all video-related applications. This limitation greatly narrows the application range of the rPPG. In this paper, a novel physiological signal preserving video compression algorithm called POSSC is proposed such that the existing rPPG signal extraction approaches can be applied directly on the compressed video without modification. The proposed approach consists of three main steps: 1) rPPG feature extraction; 2) sparse subspace clustering; and 3) region of interest (ROI)-based video compression. This paper models the skin/non-skin feature classification problem as sparse subspace clustering. Physiological signals are preserved by allocating more (fewer) bits to the ROI (non-ROI) regions. A self-collected benchmark dataset is established to evaluate the performance of POSSC in terms of body part, ROI size, light source, illumination intensity, and multiple subjects in an image. The results demonstrate that POSCC is effective in preserving physiological signals for facial videos under normal light intensity, insensitive to ROI size, shape, and the number of subjects. This paper also demonstrates the effectiveness of sparse subspace clustering for pulse region detection.

[1]  Daniel McDuff,et al.  A survey of remote optical photoplethysmographic imaging methods , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[2]  Zbignevs Marcinkevics,et al.  Imaging photoplethysmography for clinical assessment of cutaneous microcirculation at two different depths , 2016, Journal of biomedical optics.

[3]  Zhengguo Li,et al.  Region-of-Interest Based Resource Allocation for Conversational Video Communication of H.264/AVC , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[4]  Tomasz Kocejko,et al.  Proceedings of the Federated Conference on Computer Science and Information Systems pp. 405–410 ISBN 978-83-60810-22-4 Measuring Pulse Rate with a Webcam – a Non-contact Method for Evaluating Cardiac Activity , 2022 .

[5]  Westgate Road,et al.  Photoplethysmography and its application in clinical physiological measurement , 2007 .

[6]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[7]  Chun-Hsien Chou,et al.  A perceptually tuned subband image coder based on the measure of just-noticeable-distortion profile , 1994, Proceedings of 1994 IEEE International Symposium on Information Theory.

[8]  Ying Wu,et al.  Multibody Grouping by Inference of Multiple Subspaces from High-Dimensional Data Using Oriented-Frames , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Sander Stuijk,et al.  Automatic RoI Detection for Camera-Based Pulse-Rate Measurement , 2014, ACCV Workshops.

[10]  Suh-Yin Lee,et al.  Image Sensor-Based Heart Rate Evaluation From Face Reflectance Using Hilbert–Huang Transform , 2015, IEEE Sensors Journal.

[11]  Laurent Itti,et al.  Visual attention guided bit allocation in video compression , 2011, Image Vis. Comput..

[12]  Wen-Jiin Tsai,et al.  Face-Based Heart Rate Signal Decomposition and Evaluation Using Multiple Linear Regression , 2016, IEEE Sensors Journal.

[13]  Zhengguo Li,et al.  A Novel Framework for Remote Photoplethysmography Pulse Extraction on Compressed Videos , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[14]  Guoying Zhao,et al.  3D Mask Face Anti-spoofing with Remote Photoplethysmography , 2016, ECCV.

[15]  Changchen Zhao,et al.  Visual heart rate estimation and negative feedback control for fitness exercise , 2020, Biomed. Signal Process. Control..

[16]  Wen-Liang Hwang,et al.  Greedy orthogonal matching pursuit for subspace clustering to improve graph connectivity , 2018, Inf. Sci..

[17]  Daniel McDuff,et al.  The Impact of Video Compression on Remote Cardiac Pulse Measurement Using Imaging Photoplethysmography , 2017, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[18]  Xuemin Chen,et al.  Video coding using the H.264/MPEG-4 AVC compression standard , 2004, Signal Process. Image Commun..

[19]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Hagen Malberg,et al.  ROI Selection for Remote Photoplethysmography , 2013, Bildverarbeitung für die Medizin.

[21]  Ivan V. Bajic,et al.  Saliency-preserving video compression , 2011, 2011 IEEE International Conference on Multimedia and Expo.

[22]  Christine Guillemot,et al.  Perceptually-Friendly H.264/AVC Video Coding Based on Foveated Just-Noticeable-Distortion Model , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[23]  Yu Sun,et al.  Photoplethysmography Revisited: From Contact to Noncontact, From Point to Imaging , 2016, IEEE Transactions on Biomedical Engineering.

[24]  René Vidal,et al.  Sparse subspace clustering , 2009, CVPR.

[25]  Bhaskar D. Rao,et al.  Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm , 1997, IEEE Trans. Signal Process..

[26]  Daniel P. Robinson,et al.  Scalable Sparse Subspace Clustering by Orthogonal Matching Pursuit , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[28]  Jiri Matas,et al.  Non-Contact Reflectance Photoplethysmography: Progress, Limitations, and Myths , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[29]  Ivan V. Bajic,et al.  Saliency-Aware Video Compression , 2014, IEEE Transactions on Image Processing.

[30]  Ashok Veeraraghavan,et al.  DistancePPG: Robust non-contact vital signs monitoring using a camera , 2015, Biomedical optics express.

[31]  Yuting Yang,et al.  Noncontact Monitoring of Blood Oxygen Saturation Using Camera and Dual-Wavelength Imaging System , 2016, IEEE Transactions on Biomedical Engineering.

[32]  W. Verkruysse,et al.  Non-contact heart rate monitoring utilizing camera photoplethysmography in the neonatal intensive care unit - a pilot study. , 2013, Early human development.

[33]  John A. Crowe,et al.  The wavelength dependence of the photoplethysmogram and its implication to pulse oximetry , 1992, 1992 14th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[34]  Vladimir Blazek,et al.  Photoplethysmography imaging: a new noninvasive and noncontact method for mapping of the dermal perfusion changes , 2000, European Conference on Biomedical Optics.

[35]  Shuicheng Yan,et al.  Robust and Efficient Subspace Segmentation via Least Squares Regression , 2012, ECCV.

[36]  Nebojsa Jojic,et al.  -Sparse Subspace Clustering , 2016 .

[37]  Guillaume Gibert,et al.  Face detection method based on photoplethysmography , 2013, 2013 10th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[38]  Keisuke Nakamura,et al.  Real-Time Temporal Superpixels for Unsupervised Remote Photoplethysmography , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[39]  Sander Stuijk,et al.  Algorithmic Principles of Remote PPG , 2017, IEEE Transactions on Biomedical Engineering.

[40]  Sander Stuijk,et al.  Robust heart rate from fitness videos , 2017, Physiological measurement.

[41]  Sander Stuijk,et al.  Unsupervised Subject Detection via Remote PPG , 2015, IEEE Transactions on Biomedical Engineering.

[42]  Rosalind W. Picard,et al.  Non-contact, automated cardiac pulse measurements using video imaging and blind source separation , 2022 .

[43]  Sander Stuijk,et al.  Living-Skin Classification via Remote-PPG , 2017, IEEE Transactions on Biomedical Engineering.

[44]  Weisi Lin,et al.  Rate control for videophone using local perceptual cues , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[45]  Yoshinori Kuno,et al.  Robust Heart Rate Measurement from Video Using Select Random Patches , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[46]  Nicu Sebe,et al.  Self-Adaptive Matrix Completion for Heart Rate Estimation from Face Videos under Realistic Conditions , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).