Simultaneous detection of blink and heart rate using multi-channel ICA from smart phone videos

Abstract Physiological indexes, such as blink frequency and heart rate, express the physical and mental state of a human. This paper presents an algorithm for simultaneous detection of eye blink and heart rate using multi-channel ICA. Video sequences captured from smart phones are used in which subjects are requested to sit without big motion. Statistic information from the R, G and B components of eyes and their surrounding facial region is explored to discriminate the different sources that mixed in each image. The proposed algorithm extracts both eye blink and cardiac signal as different sources at the same time by 6-channel SOBI without any other complex processing. Meanwhile, a kurtosis based method is proposed to automatically select blink and cardiac signals from the output separations. Different ICA algorithms and channel numbers as well as a series of head moving modes are employed to test the robustness and accuracy of the algorithm. Experiments on twenty subjects show that multi-channel ICA is capable of precisely separating the eye blink and cardiac signals in a less complex way.

[1]  Antoine Picot,et al.  On-Line Detection of Drowsiness Using Brain and Visual Information , 2012, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[2]  Gerard de Haan,et al.  Robust Pulse Rate From Chrominance-Based rPPG , 2013, IEEE Transactions on Biomedical Engineering.

[3]  Sepideh Hajipour Sardouie,et al.  An Efficient Jacobi-Like Deflationary ICA Algorithm: Application to EEG Denoising , 2015, IEEE Signal Processing Letters.

[4]  Amit Konar,et al.  Electro-oculogram Based Classification of Eye Movement Direction , 2012, ACITY.

[5]  Kilseop Ryu,et al.  Evaluation of mental workload with a combined measure based on physiological indices during a dual task of tracking and mental arithmetic , 2005 .

[6]  Terrence J. Sejnowski,et al.  Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources , 1999, Neural Computation.

[7]  Antoine Souloumiac,et al.  Jacobi Angles for Simultaneous Diagonalization , 1996, SIAM J. Matrix Anal. Appl..

[8]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[9]  Neil Cooke,et al.  VOG-enhanced ICA for SSVEP response detection from consumer-grade EEG , 2014, 2014 22nd European Signal Processing Conference (EUSIPCO).

[10]  Eric Moulines,et al.  A blind source separation technique using second-order statistics , 1997, IEEE Trans. Signal Process..

[11]  Vasilis Ntziachristos,et al.  Looking and listening to light: the evolution of whole-body photonic imaging , 2005, Nature Biotechnology.

[12]  Shinobu Tanaka,et al.  Comparison between red, green and blue light reflection photoplethysmography for heart rate monitoring during motion , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[13]  Sijung Hu,et al.  The preliminary investigation of imaging photoplethysmographic system , 2007 .

[14]  A. Hyvarinen,et al.  Fast ICA for noisy data using Gaussian moments , 1999, ISCAS'99. Proceedings of the 1999 IEEE International Symposium on Circuits and Systems VLSI (Cat. No.99CH36349).

[15]  C. Takano,et al.  Heart rate measurement based on a time-lapse image. , 2007, Medical engineering & physics.

[16]  L. O. Svaasand,et al.  Remote plethysmographic imaging using ambient light. , 2008, Optics express.

[17]  Margrit Betke,et al.  Real Time Eye Tracking and Blink Detection with USB Cameras , 2005 .

[18]  A. B. Hertzman.  THE BLOOD SUPPLY OF VARIOUS SKIN AREAS AS ESTIMATED BY THE PHOTOELECTRIC PLETHYSMOGRAPH , 1938 .

[19]  Yu Sun,et al.  Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise. , 2011, Journal of biomedical optics.

[20]  Anton Vonk-Noordegraaf,et al.  Stroke volume response during exercise measured by acetylene uptake and MRI , 2007, Physiological measurement.

[21]  Xianta Jiang,et al.  Capturing and Evaluating Blinks from Video-based Eyetrackers Developed a Data-driven Eye-blink Detection Algorithm for 50-hz Eyetracking Proto- Cols, Where the Pupil Diameter Was Used for Correcting , 2022 .

[22]  Wen Jun Jiang,et al.  Real-time quantifying heart beat rate from facial video recording on a smart phone using Kalman filters , 2014, 2014 IEEE 16th International Conference on e-Health Networking, Applications and Services (Healthcom).

[23]  Westgate Road,et al.  Photoplethysmography and its application in clinical physiological measurement , 2007 .

[24]  Sandra P Marshall,et al.  Identifying cognitive state from eye metrics. , 2007, Aviation, space, and environmental medicine.

[25]  Xingfu Zhang,et al.  Two Dimensional Principal Component Analysis based Independent Component Analysis for face recognition , 2011, 2011 International Conference on Multimedia Technology.

[26]  Frédo Durand,et al.  Eulerian video magnification for revealing subtle changes in the world , 2012, ACM Trans. Graph..

[27]  Izabela Rejer,et al.  Benefits of ICA in the Case of a Few Channel EEG , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[28]  Daniel McDuff,et al.  Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam , 2011, IEEE Transactions on Biomedical Engineering.

[29]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[30]  Jia Wu,et al.  An implementation of independent component analysis for 3D statistical shape analysis , 2014, Biomed. Signal Process. Control..

[31]  Te-Won Lee.  Independent Component Analysis , 1998, Springer US.

[32]  Hagen Malberg,et al.  Improved heart rate detection for camera-based photoplethysmography by means of Kalman filtering , 2015, 2015 IEEE 35th International Conference on Electronics and Nanotechnology (ELNANO).

[33]  Jon Atli Benediktsson,et al.  Hyperspectral Image Classification With Independent Component Discriminant Analysis , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[34]  Simone Benedetto,et al.  Driver workload and eye blink duration , 2011 .

[35]  K. Banitsas,et al.  A novel method to detect Heart Beat Rate using a mobile phone , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[36]  Daniel McDuff,et al.  Remote Detection of Photoplethysmographic Systolic and Diastolic Peaks Using a Digital Camera , 2014, IEEE Transactions on Biomedical Engineering.

[37]  B. Li,et al.  Non-contact detection of oxygen saturation based on visible light imaging device using ambient light. , 2013, Optics express.