Emotional State Recognition Using Facial Expression, Voice, and Physiological Signal

Since twenty years, the computer modelling of emotion is a theme increasingly recognized, particularly in the field of human-machine interaction.2 The term “emotion” is relatively difficult to define from a scientific point of view. Indeed, the phenomenon of emotion is based at the same time on physical, physiological, mental and behavioural considerations. Thus, many areas such as affective computing and image processing are interested in human emotional dimensions. The growing maturity of the field of emotion recognition is creating new needs in terms of engineering. After a replication phase, during which numerous works have been proposed with recognition systems,3 we are gradually entering an empiricism phase,4 where models for the design are developed.3 Most designed systems allow passive recognition of emotions. To define emotion, we base ourselves on Scherer’s theory.5 An emotion is characterized by a highly synchronized expression: the whole body (face, limbs, and physiological reactions) reacts in unison and the human emotional expression is clearly multimodal. Indeed, a large number of studies have been carried out in order to define the relationship between emotion and physiological signal. These have allowed highlighting a significant correlation between this type of signal and certain emotional states.