Knowledge Engineering Aspects of Affective Bi-Modal Educational Applications

This paper analyses the knowledge and software engineering aspects of educational applications that provide affective bi-modal human-computer interaction. For this purpose, a system that provides affective interaction based on evidence from two different modes has been developed. More specifically, the system’s inferences about students’ emotions are based on user input evidence from the keyboard and the microphone. Evidence from these two modes is combined by a user modelling component that incorporates user stereotypes as well as a multi criteria decision making theory. The mechanism that integrates the inferences from the two modes has been based on the results of two empirical studies that were conducted in the context of knowledge engineering of the system. The evaluation of the developed system showed significant improvements in the recognition of the emotional states of users.

[1]  Hillary Anger Elfenbein,et al.  When familiarity breeds accuracy: cultural exposure and facial emotion recognition. , 2003, Journal of personality and social psychology.

[2]  Jonathan Klein,et al.  Computers that recognise and respond to user emotion: theoretical and practical implications , 2002, Interact. Comput..

[3]  Peter C. Fishburn,et al.  Letter to the Editor - Additive Utilities with Incomplete Product Sets: Application to Priorities and Assignments , 1967, Oper. Res..

[4]  Alfred Kobsa,et al.  Personalised hypermedia presentation techniques for improving online customer relationships , 2001, The Knowledge Engineering Review.

[5]  L. Rothkrantz,et al.  Toward an affect-sensitive multimodal human-computer interaction , 2003, Proc. IEEE.

[6]  Judy Kay,et al.  Intelligent Tutoring Systems , 2000, Lecture Notes in Computer Science.

[7]  Sharon Oviatt,et al.  User-centered modeling and evaluation of multimodal interfaces , 2003, Proc. IEEE.

[8]  Elaine Rich.  Users are individuals: individualizing user models , 1999, Int. J. Hum. Comput. Stud..

[9]  Ching-Lai Hwang,et al.  Multiple Attribute Decision Making: Methods and Applications - A State-of-the-Art Survey , 1981, Lecture Notes in Economics and Mathematical Systems.

[10]  Eva Hudlicka,et al.  To feel or not to feel: The role of affect in human-computer interaction , 2003, Int. J. Hum. Comput. Stud..

[11]  Tsuyoshi Moriyama,et al.  Measurement of human vocal emotion using fuzzy control , 2001, Systems and Computers in Japan.

[12]  D. Goleman.  Emotional Intelligence: Why It Can Matter More Than IQ , 1995 .

[13]  H. Saito,et al.  Evaluation of the relation between emotional concepts and emotional parameters in speech , 2001 .

[14]  Rosalind W. Picard.  Affective computing: challenges , 2003, Int. J. Hum. Comput. Stud..

[15]  G.A. Tsihrintzis,et al.  Detection and expression classification systems for face images (FADECS) , 2005, IEEE Workshop on Signal Processing Systems Design and Implementation, 2005..

[16]  Ching-Lai Hwang,et al.  Fuzzy Multiple Attribute Decision Making - Methods and Applications , 1992, Lecture Notes in Economics and Mathematical Systems.