Detection of physical stress using multispectral imaging

Abstract This study explored the potential of multispectral imaging for detecting physical stress on humans. Multispectral images were obtained from recruited participants, and a max completion-based differential tissue oxygen saturation (StO2) algorithm model was proposed and established without background information. Further correlation analysis showed that multi-subject facial StO2 fluctuations have similar intra-class affective patterns under physical stress. The algorithm model was verified with respect to physical stress ground truth to classify the baseline and physical stress status. The algorithm implements non-contact, non-intervention, and background-free physical stress detection. The algorithm achieved promising results in the experiment with an accurate rate of higher than 90%, which will provide an excellent foundation for future industrialization. Experimental results demonstrated that multispectral imaging, as a non-invasive method, has the potential to identify physical stress on humans.

[1]  Gamal ElMasry,et al.  Quality classification of cooked, sliced turkey hams using NIR hyperspectral imaging system , 2011 .

[2]  Shadi AlZu'bi,et al.  Multiresolution Analysis Using Wavelet, Ridgelet, and Curvelet Transforms for Medical Image Segmentation , 2011, Int. J. Biomed. Imaging.

[3]  Ioannis T. Pavlidis,et al.  Thermal image analysis for anxiety detection , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[4]  A. Merla,et al.  Mother and child in synchrony: Thermal facial imprints of autonomic contagion , 2012, Biological Psychology.

[5]  Reuben A. Farrugia,et al.  Driver fatigue monitoring system using Support Vector Machines , 2012, 2012 5th International Symposium on Communications, Control and Signal Processing.

[6]  M. Otto Physical stress and bacterial colonization. , 2014, FEMS microbiology reviews.

[7]  Huosheng Hu,et al.  A Novel Real-Time Face Tracking Algorithm for Detection of Driver Fatigue , 2010, 2010 Third International Symposium on Intelligent Information Technology and Security Informatics.

[8]  M. Baum,et al.  Prenatal Programming of Hypertension Induces Sympathetic Overactivity in Response to Physical Stress , 2013, Hypertension.

[9]  Clement Atzberger,et al.  New ways to extract archaeological information from hyperspectral pixels , 2014 .

[10]  Thomas B. Moeslund,et al.  Facial video-based detection of physical fatigue for maximal muscle activity , 2016, IET Comput. Vis..

[11]  Enzo Pasquale Scilingo,et al.  Recognizing Emotions Induced by Affective Sounds through Heart Rate Variability , 2015, IEEE Transactions on Affective Computing.

[12]  F. Jones,et al.  Stress: Myth, Theory and Research , 2001 .

[13]  Kaizhu Huang,et al.  Siamese network ensemble for visual tracking , 2018, Neurocomputing.

[14]  Lisa Dorn,et al.  Stress, fatigue, health, and risk of road traffic accidents among professional drivers: the contribution of physical inactivity. , 2006, Annual review of public health.

[15]  A. Berstad,et al.  The influence of prolonged physical stress on gastric juice components in healthy man. , 1988, Scandinavian journal of gastroenterology.

[16]  Thomas B. Moeslund,et al.  Contactless measurement of muscles fatigue by tracking facial feature points in a video , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[17]  Jean-Luc Starck,et al.  3D curvelet transforms and astronomical data restoration , 2010, Applied and Computational Harmonic Analysis.

[18]  Marc Garbey,et al.  Interacting with human physiology , 2007, Comput. Vis. Image Underst..

[19]  M. Gladwin,et al.  Noninvasive Determination of Spatially Resolved and Time-Resolved Tissue Perfusion in Humans During Nitric Oxide Inhibition and Inhalation by Use of a Visible-Reflectance Hyperspectral Imaging Technique , 2001, Circulation.

[20]  Ying Gao,et al.  Affective Assessment by Digital Processing of the Pupil Diameter , 2013, IEEE Transactions on Affective Computing.

[21]  M. Gladwin,et al.  Imaging hemoglobin oxygen saturation in sickle cell disease patients using noninvasive visible reflectance hyperspectral techniques: effects of nitric oxide. , 2003, American journal of physiology. Heart and circulatory physiology.

[22]  J. Mansfield,et al.  Hyperspectral imaging: a new approach to the diagnosis of hemorrhagic shock. , 2006, The Journal of trauma.

[23]  Zhiwei Zhu,et al.  Real-time nonintrusive monitoring and prediction of driver fatigue , 2004, IEEE Transactions on Vehicular Technology.

[24]  Rosemarie Velik,et al.  Impact of Mental and Physical Stress on Blood Pressure and Pulse Pressure under Normobaric versus Hypoxic Conditions , 2014, PloS one.

[25]  Feiping Nie,et al.  Robust Object Co-Segmentation Using Background Prior , 2018, IEEE Transactions on Image Processing.

[26]  S. Skinner,et al.  The circulation in forearm skin and muscle during adrenaline infusions. , 1962, The Australian journal of experimental biology and medical science.

[27]  Aksone Nouvong,et al.  Hyperspectral Imaging in Diabetic Foot Wound Care , 2010, Journal of diabetes science and technology.

[28]  A. Nolan,et al.  The recovery of human saliva using the Salivette system. , 1994, European journal of clinical chemistry and clinical biochemistry : journal of the Forum of European Clinical Chemistry Societies.

[29]  Yong Zhao,et al.  A practical driver fatigue detection algorithm based on eye state , 2010, 2010 Asia Pacific Conference on Postgraduate Research in Microelectronics and Electronics (PrimeAsia).

[30]  O. W. van Assendelft,et al.  Spectrophotometry of haemoglobin derivatives , 1970 .

[31]  Dong Xu,et al.  Advanced Deep-Learning Techniques for Salient and Category-Specific Object Detection: A Survey , 2018, IEEE Signal Processing Magazine.

[32]  Ilkka Pölönen,et al.  Detecting field cancerization using a hyperspectral imaging system , 2013, Lasers in surgery and medicine.

[33]  Frédo Durand,et al.  Eulerian video magnification for revealing subtle changes in the world , 2012, ACM Trans. Graph..

[34]  D. Tschoepe,et al.  Effect of mental and physical stress on platelet activation markers in depressed patients and healthy subjects: a pilot study , 2004, Psychiatry Research.

[35]  Arman Savran,et al.  Temporal Bayesian Fusion for Affect Sensing: Combining Video, Audio, and Lexical Modalities , 2015, IEEE Transactions on Cybernetics.

[36]  Vitold E. Pozhar,et al.  Compact AOTF-based spectral imaging system for medical endoscopic analysis , 2013 .

[37]  Torsten Lehmann,et al.  Open platform, eight-channel, portable bio-potential and activity data logger for wearable medical device development , 2015 .

[38]  Qingshan Liu,et al.  Learning Multiscale Active Facial Patches for Expression Analysis , 2015, IEEE Transactions on Cybernetics.

[39]  Tommi S. Jaakkola,et al.  Weighted Low-Rank Approximations , 2003, ICML.

[40]  J. VanMeter,et al.  Lying about facial recognition: An fMRI study , 2009, Brain and Cognition.

[41]  David Sander,et al.  Thermal Analysis of Facial Muscles Contractions , 2011, IEEE Transactions on Affective Computing.

[42]  Kevin A. Johnson,et al.  Detecting Deception Using Functional Magnetic Resonance Imaging , 2005, Biological Psychiatry.

[43]  Kan Hong,et al.  Remote sensing of stress using electro-optics imaging technique , 2009, Security + Defence.

[44]  Carl B. Cross,et al.  Thermal imaging to detect physiological indicators of stress in humans , 2013, Defense, Security, and Sensing.

[45]  Yann LeCun,et al.  Learning a similarity metric discriminatively, with application to face verification , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[46]  Aouatif Amine,et al.  Driver's fatigue detection based on yawning extraction , 2014 .

[47]  J Grayson,et al.  The effect of adrenaline and noradrenaline on coronary vascular reserve in the dog. , 1980, The Journal of physiology.

[48]  Arcangelo Merla,et al.  Exploring the Use of Thermal Infrared Imaging in Human Stress Research , 2014, PloS one.

[49]  Sheng Hong,et al.  Real-time stress assessment using thermal imaging , 2016, The Visual Computer.

[50]  K. Norozi,et al.  Physical stress testing of bovine jugular veins using magnetic resonance imaging, echocardiography and electrical velocimetry. , 2010, Interactive cardiovascular and thoracic surgery.

[51]  Emmanuel J. Candès,et al.  The curvelet transform for image denoising , 2002, IEEE Trans. Image Process..

[52]  Dvijesh Shastri,et al.  Imaging Facial Signs of Neurophysiological Responses , 2009, IEEE Transactions on Biomedical Engineering.

[53]  Takayuki Okatani,et al.  Efficient algorithm for low-rank matrix factorization with missing components and performance comparison of latest algorithms , 2011, 2011 International Conference on Computer Vision.

[54]  Yong He,et al.  Detection of early blight and late blight diseases on tomato leaves using hyperspectral imaging , 2015, Scientific Reports.

[55]  Wanhui Wen,et al.  Emotion Recognition Based on Multi-Variant Correlation of Physiological Signals , 2014, IEEE Transactions on Affective Computing.

[56]  Gerold Baier,et al.  A Multivariate Approach to Correlation Analysis Based on Random Matrix Theory , 2008 .

[57]  A. Raut,et al.  Effect of Withania somnifera on physical and cardiovascular performance induced by physical stress in healthy human volunteers , 2016 .

[58]  Marc Garbey,et al.  Contact-Free Measurement of Cardiac Pulse Based on the Analysis of Thermal Imagery , 2007, IEEE Transactions on Biomedical Engineering.

[59]  Budi Sumanto,et al.  Stress level detection using heart rate, blood pressure, and GSR and stress therapy by utilizing infrared , 2015, 2015 International Conference on Industrial Instrumentation and Control (ICIC).

[60]  K. Schomacker,et al.  Assessing diabetic foot ulcer development risk with hyperspectral tissue oximetry. , 2011, Journal of biomedical optics.

[61]  Vittorio Gallese,et al.  The Autonomic Signature of Guilt in Children: A Thermal Infrared Imaging Study , 2013, PloS one.

[62]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[63]  Alexandre Bernardino,et al.  Matrix Completion for Weakly-Supervised Multi-Label Image Classification , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[64]  P Hjemdahl,et al.  Effects of mental and physical stress on platelet function in patients with stable angina pectoris and healthy controls. , 1997, European heart journal.

[65]  M. Irfan,et al.  Physical stress may result in growth suppression and pubertal delay in working boys , 2012 .

[66]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[67]  H. Izumi,et al.  Circulating adrenaline released by sympathoadrenal activation elicits acute vasodilatation in the rat masseter muscle. , 2009, Archives of oral biology.

[68]  U. Soori,et al.  Assessment of Tissue Blood Perfusion In-Vitro Using Hyperspectral and Thermal Imaging Techniques , 2011, 2011 5th International Conference on Bioinformatics and Biomedical Engineering.

[69]  Lei Guo,et al.  When Deep Learning Meets Metric Learning: Remote Sensing Image Scene Classification via Learning Discriminative CNNs , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[70]  Peng Sun,et al.  Drowsiness Detection Based on Eyelid Movement , 2010, 2010 Second International Workshop on Education Technology and Computer Science.

[71]  Miguel Torres-Torriti,et al.  Face salient points and eyes tracking for robust drowsiness detection , 2012, Robotica.

[72]  Soon Kwon,et al.  A novel driver fatigue monitoring using optical imaging of face on safe driving system , 2016, 2016 International Conference on Control, Electronics, Renewable Energy and Communications (ICCEREC).

[73]  Amita M Pardeshi,et al.  Comparison of Anthropometric Parameters and Blood Pressure Changes in Response to Physical Stress Test in Normotensive Subjects with or Without Family History of Hypertension. , 2016, Indian journal of physiology and pharmacology.

[74]  Ioannis Pavlidis,et al.  Seeing through the face of deception , 2002, Nature.

[75]  Mark Hallett,et al.  Alteration of brain functional network at rest and in response to YMCA physical stress test in concussed athletes: RsFMRI study , 2011, NeuroImage.

[76]  Junwei Han,et al.  A Unified Metric Learning-Based Framework for Co-Saliency Detection , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[77]  Fillia Makedon,et al.  Learning from Incomplete Ratings Using Non-negative Matrix Factorization , 2006, SDM.

[78]  Sildomar T. Monteiro,et al.  Consistency of Measurements of Wavelength Position From Hyperspectral Imagery: Use of the Ferric Iron Crystal Field Absorption at $\sim$900 nm as an Indicator of Mineralogy , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[79]  Ioannis Pavlidis Continuous physiological monitoring , 2003, Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE Cat. No.03CH37439).

[80]  Fei Liu,et al.  [Study of detection of SPAD value in tomato leaves stressed by grey mold based on hyperspectral technique]. , 2012, Guang pu xue yu guang pu fen xi = Guang pu.

[81]  W. Gillan,et al.  Correlations among Stress, Physical Activity and Nutrition: School Employee Health Behavior. , 2013 .

[82]  A. Merla,et al.  Mom feels what her child feels: thermal signatures of vicarious autonomic response while watching children in a stressful situation , 2013, Front. Hum. Neurosci..

[83]  R.M. Manjeshwar,et al.  Quantitative fluoroscopic dose saving in cardiovascular imaging with a novel motion discriminating temporal filter , 2005, Computers in Cardiology, 2005.

[84]  Sheng Hong,et al.  Classification of the emotional stress and physical stress using signal magnification and canonical correlation analysis , 2018, Pattern Recognit..

[85]  Qing Chen,et al.  Primary immune surveillance: some like it hot , 2007, Journal of Molecular Medicine.

[86]  K. Zuzak,et al.  Visible reflectance hyperspectral imaging: characterization of a noninvasive, in vivo system for determining tissue perfusion. , 2002, Analytical chemistry.