A Multi-Modal Emotion-Diagnosis System to Support e-Learning

There has been a considerable amount of research done into the detection and evaluation of human emotions from implicit communication channels, including facial expressions. However, most studies have extracted facial features for some specific emotions in specific situations. This paper describes a system which can judge the emotions of an e-Learning user from his/her facial expression and biometrical signals. Criteria for classifying eight emotions were established using a time sequential subjective evaluation of the subject's emotions as well as the time sequential analysis of the subject's facial expressions and biometrical signals. The coincidence ratio between the discriminated emotions based upon the criteria of emotion diagnosis and the time sequential subjective evaluation of emotions for 10 e-Learning subjects was 74%. When only the facial expressions were taken into account, the coincidence ratio was 68%. These results confirm the effectiveness of the multi-modal emotion diagnosis