Detecting Happiness Using Hyperspectral Imaging Technology

Hyperspectral imaging (HSI) technology can be used to detect human emotions based on the power of material discrimination from their faces. In this paper, HSI is used to remotely sense and distinguish blood chromophores in facial tissues and acquire an evaluation indicator (tissue oxygen saturation, StO2) using an optical absorption model. This study explored facial analysis while people were showing spontaneous expressions of happiness during social interaction. Happiness, as a psychological emotion, has been shown to be strongly linked to other activities such as physiological reaction and facial expression. Moreover, facial expression as a communicative motor behavior likely arises from musculoskeletal anatomy, neuromuscular activity, and individual personality. This paper quantified the neuromotor movements of tissues surrounding some regions of interest (ROIs) on smiling happily. Next, we selected six regions—the forehead, eye, nose, cheek, mouth, and chin—according to a facial action coding system (FACS). Nineteen segments were subsequently partitioned from the above ROIs. The affective data (StO2) of 23 young adults were acquired by HSI while the participants expressed emotions (calm or happy), and these were used to compare the significant differences in the variations of StO2 between the different ROIs through repeated measures analysis of variance. Results demonstrate that happiness causes different distributions in the variations of StO2 for the above ROIs; these are explained in depth in the article. This study establishes that facial tissue oxygen saturation is a valid and reliable physiological indicator of happiness and merits further research.

[1]  Filippo Zappasodi,et al.  The cerebral correlates of subliminal emotions: an electroencephalographic study with emotional hybrid faces , 2015, The European journal of neuroscience.

[2]  Steven A. Shafer,et al.  Using color to separate reflection components , 1985 .

[3]  J V Basmajian,et al.  Electromyographic investigation of procerus and frontalis muscles. , 1976, Electromyography and clinical neurophysiology.

[4]  J. N. Bassili Emotion recognition: the role of facial movement and the relative importance of upper and lower areas of the face. , 1979, Journal of personality and social psychology.

[5]  Matthew Witt,et al.  Laypeople's preferences regarding frontal dentofacial esthetics: periodontal factors. , 2011, Journal of the American Dental Association.

[6]  Maurizio Codispoti,et al.  Watching emotional movies: affective reactions and gender differences. , 2008, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[7]  J. Gross,et al.  Emotion elicitation using films , 1995 .

[8]  J. Persing,et al.  Perception of Age, Attractiveness, and Tiredness After Isolated and Combined Facial Subunit Aging , 2015, Aesthetic Plastic Surgery.

[9]  Tamás D. Gedeon,et al.  Automatic Group Happiness Intensity Analysis , 2015, IEEE Transactions on Affective Computing.

[10]  S. Lyubomirsky,et al.  The Benefits of Frequent Positive Affect: Does Happiness Lead to Success? , 2005, Psychological bulletin.

[11]  Noel D.G. White,et al.  Detection of fungal infection and Ochratoxin A contamination in stored barley using near-infrared hyperspectral imaging , 2016 .

[12]  Charles J. Holahan,et al.  Finding happiness in negative emotions: An experimental test of a novel expressive writing paradigm , 2011 .

[13]  Gary A. Shaw,et al.  Spectral Imaging for Remote Sensing , 2003 .

[14]  Emily Mower Provost,et al.  Emotion Recognition During Speech Using Dynamics of Multiple Regions of the Face , 2015, ACM Trans. Multim. Comput. Commun. Appl..

[15]  U. Soori,et al.  Assessment of Tissue Blood Perfusion In-Vitro Using Hyperspectral and Thermal Imaging Techniques , 2011, 2011 5th International Conference on Bioinformatics and Biomedical Engineering.

[16]  Filippo Zappasodi,et al.  Contrasting Hemispheric Asymmetries for Emotional Processing From Event-Related Potentials and Behavioral Responses , 2018, Neuropsychology.

[17]  A. Lipsitz,et al.  Research Methods with a Smile: A Gender Difference Exercise that Teaches Methodology , 2000 .

[18]  Sylvia D. Kreibig,et al.  Autonomic nervous system activity in emotion: A review , 2010, Biological Psychology.

[19]  Sabine C. Herpertz,et al.  Facial Reactions during Emotion Recognition in Borderline Personality Disorder: A Facial Electromyography Study , 2013, Psychopathology.

[20]  Reyer Zwiggelaar,et al.  Thermal Facial Analysis for Deception Detection , 2014, IEEE Transactions on Information Forensics and Security.

[21]  R. Gur,et al.  Differences in facial expressions of four universal emotions , 2004, Psychiatry Research.

[22]  D. Tranel,et al.  Gastric myoelectrical activity as an index of emotional arousal. , 2006, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[23]  A. Berardelli,et al.  Facial bradykinesia , 2012, Journal of Neurology, Neurosurgery & Psychiatry.

[24]  Marc B Ackerman,et al.  Hard- and soft-tissue contributions to the esthetics of the posed smile in growing patients seeking orthodontic treatment. , 2008, American journal of orthodontics and dentofacial orthopedics : official publication of the American Association of Orthodontists, its constituent societies, and the American Board of Orthodontics.

[25]  M. Gamer,et al.  Patterns of Sympathetic Responses Induced by Different Stress Tasks , 2008, The open neurology journal.

[26]  Cary L. Cooper,et al.  Emotions at work : theory, research and applications in management , 2001 .

[27]  Guilherme Janson,et al.  Influence of orthodontic treatment, midline position, buccal corridor and smile arc on smile attractiveness. , 2011, The Angle orthodontist.

[28]  Sharifah Mumtazah Syed Ahmad,et al.  Smile detection using hybrid face representation , 2016, J. Ambient Intell. Humaniz. Comput..

[29]  Mark A. Richardson,et al.  An introduction to hyperspectral imaging and its application for security, surveillance and target acquisition , 2010 .

[30]  G. Gendolla,et al.  The informational impact of mood on effort mobilization: a study of cardiovascular and electrodermal responses. , 2001, Emotion.

[31]  Nor Nazrina Mohamad Nazry,et al.  Mood and learning in navigation-based serious games , 2017, Comput. Hum. Behav..

[32]  L Atkinson,et al.  Alexithymia: a comparative study of three self-report measures. , 1988, Journal of psychosomatic research.

[33]  Francesca Benuzzi,et al.  Fear and happiness in the eyes: An intra-cerebral event-related potential study from the human amygdala , 2012, Neuropsychologia.

[34]  Urs Fischer,et al.  Uncrossed cortico-muscular projections in humans are abundant to facial muscles of the upper and lower face, but may differ between sexes , 2005, Journal of Neurology.

[35]  A. Lindell,et al.  Lateralization of the expression of facial emotion in humans. , 2018, Progress in brain research.

[36]  K. Schomacker,et al.  Assessing diabetic foot ulcer development risk with hyperspectral tissue oximetry. , 2011, Journal of biomedical optics.

[37]  P. Ekman,et al.  The Duchenne smile: emotional expression and brain physiology. II. , 1990, Journal of personality and social psychology.

[38]  Wanhui Wen,et al.  Emotion Recognition Based on Multi-Variant Correlation of Physiological Signals , 2014, IEEE Transactions on Affective Computing.

[39]  E. Ross,et al.  Decoding facial blends of emotion: Visual field, attentional and hemispheric biases , 2013, Brain and Cognition.

[40]  Marc Mehu,et al.  Naturalistic observations of smiling and laughter in human group interactions , 2008 .

[41]  H. Critchley,et al.  From facial mimicry to emotional empathy: A role for norepinephrine? , 2010, Social neuroscience.

[42]  Cigdem Eroglu Erdem,et al.  BAUM-1: A Spontaneous Audio-Visual Face Database of Affective and Mental States , 2017, IEEE Transactions on Affective Computing.

[43]  P. Najt,et al.  Models of hemispheric specialization in facial emotion perception--a reevaluation. , 2013, Emotion.

[44]  N. Rosenthal,et al.  Treatment of depression with onabotulinumtoxinA: a randomized, double-blind, placebo controlled trial. , 2014, Journal of psychiatric research.

[45]  P. Ekman,et al.  Nonverbal Leakage and Clues to Deception †. , 1969, Psychiatry.

[46]  I. Pavlidis,et al.  Fast by Nature - How Stress Patterns Define Human Experience and Performance in Dexterous Tasks , 2012, Scientific Reports.

[47]  Bianca I. Knoll,et al.  The Influence of Forehead, Brow, and Periorbital Aesthetics on Perceived Expression in the Youthful Face , 2008, Plastic and reconstructive surgery.

[48]  Qi Wu,et al.  A False Trail to Follow: Differential Effects of the Facial Feedback Signals From the Upper and Lower Face on the Recognition of Micro-Expressions , 2018, Front. Psychol..

[49]  P. Ekman,et al.  A tool for the analysis of motion picture film or video tape. , 1969, The American psychologist.

[50]  J. Russell A circumplex model of affect. , 1980 .

[51]  W R Proffit,et al.  The emerging soft tissue paradigm in orthodontic diagnosis and treatment planning. , 1999, Clinical orthodontics and research.

[52]  Michèle Desjardins,et al.  Preliminary investigation of multispectral retinal tissue oximetry mapping using a hyperspectral retinal camera. , 2016, Experimental eye research.

[53]  Matthew Witt,et al.  Laypeople's preferences regarding frontal dentofacial esthetics: tooth-related factors. , 2011, Journal of the American Dental Association.

[54]  G J Taylor,et al.  Toward the development of a new self-report alexithymia scale. , 1985, Psychotherapy and psychosomatics.

[55]  Naoyuki Hayashi,et al.  Regional differences in facial skin blood flow responses to the cold pressor and static handgrip tests , 2012, European Journal of Applied Physiology.

[56]  Guangyuan Liu,et al.  Detection of Psychological Stress Using a Hyperspectral Imaging Technique , 2014, IEEE Transactions on Affective Computing.

[57]  Peter G. Rendell,et al.  Disgust, but not anger provocation, enhances levator labii superioris activity during exposure to moral transgressions , 2014, Biological Psychology.

[58]  Sridhar Radhakrishnan,et al.  Neurophysiology of spontaneous facial expressions: I. Motor control of the upper and lower face is behaviorally independent in adults , 2016, Cortex.

[59]  P. Ekman Facial expressions of emotion: an old controversy and new findings. , 1992, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[60]  S. A. Neumann,et al.  Similar patterns of cardiovascular response during emotional activation as a function of affective valence and arousal and gender. , 2001, Journal of psychosomatic research.

[61]  Marc Garbey,et al.  Interacting with human physiology , 2007, Comput. Vis. Image Underst..

[62]  P. Lagacherie,et al.  Using legacy data for correction of soil surface clay content predicted from VNIR/SWIR hyperspectral airborne images , 2016 .

[63]  Wanhui Wen,et al.  Analysis of affective ECG signals toward emotion recognition , 2010 .

[64]  H. Kiyak,et al.  Comparing the perception of dentists and lay people to altered dental esthetics. , 1999, Journal of esthetic dentistry.

[65]  Bernhard Dahme,et al.  Emotions and airway resistance in asthma: study with whole body plethysmography. , 2005, Psychophysiology.

[66]  Alessandro Angrilli,et al.  Gender differences in emotional responses: A psychophysiological study , 2012, Physiology & Behavior.

[67]  Ravi Bansal,et al.  Using the Circumplex Model of Affect to Study Valence and Arousal Ratings of Emotional Faces by Children and Adults with Autism Spectrum Disorders , 2014, Journal of autism and developmental disorders.

[68]  María Malfaz,et al.  A New Approach to Modeling Emotions and Their Use on a Decision-Making System for Artificial Agents , 2012, IEEE Transactions on Affective Computing.

[69]  K. Zuzak,et al.  Visible reflectance hyperspectral imaging: characterization of a noninvasive, in vivo system for determining tissue perfusion. , 2002, Analytical chemistry.