Development process of an affective bi-modal Intelligent Tutoring System

This paper presents the development process of an Intelligent Tutoring System that performs emotion recognition on the basis of two modalities: keyboard and microphone. The system uses a multi-criteria theory to combine information about the user emotions through the two modalities. In this paper we focus on presenting and discussing the experimental studies that were conducted for the purposes of the design of the system. The experimental studies yield results concerning the way that human observers perform emotion recognition on other humans and then we show how we use these results to design the reasoning mechanism of the Intelligent Tutoring system that performs emotion recognition.

[1]  Philippe Vincke,et al.  Multicriteria Decision-aid , 1993 .

[2]  Zhiwei Zhu,et al.  Toward a decision-theoretic framework for affect recognition and user assistance , 2006, Int. J. Hum. Comput. Stud..

[3]  Richard J. Davidson,et al.  Parsing the subcomponents of emotion and disorders of emotion: Perspectives from affective neuroscience. , 2003 .

[4]  Felix Naumann.  Data Fusion and Data Quality , 1998 .

[5]  Zhigang Deng,et al.  Analysis of emotion recognition using facial expressions, speech and multimodal information , 2004, ICMI '04.

[6]  Sharon Oviatt,et al.  User-centered modeling and evaluation of multimodal interfaces , 2003, Proc. IEEE.

[7]  K. Scherer,et al.  Handbook of affective sciences. , 2003 .

[8]  Tsutomu Miyasato,et al.  Bimodal Emotion Recognition by Man and Machine , 2007 .

[9]  Elaine Rich,et al.  Users are Individuals: Individualizing User Models , 1999, Int. J. Man Mach. Stud..

[10]  L. Rothkrantz,et al.  Toward an affect-sensitive multimodal human-computer interaction , 2003, Proc. IEEE.

[11]  Elaine Rich.  Users are individuals: individualizing user models , 1999, Int. J. Hum. Comput. Stud..

[12]  Evangelos Triantaphyllou,et al.  An examination of the effectiveness of multi-dimensional decision-making methods: A decision-making paradox , 1989, Decis. Support Syst..

[13]  Kristina Höök,et al.  Evaluating affective interactions , 2007, Int. J. Hum. Comput. Stud..

[14]  G.A. Tsihrintzis,et al.  Detection and expression classification systems for face images (FADECS) , 2005, IEEE Workshop on Signal Processing Systems Design and Implementation, 2005..

[15]  Ching-Lai Hwang,et al.  Fuzzy Multiple Attribute Decision Making - Methods and Applications , 1992, Lecture Notes in Economics and Mathematical Systems.

[16]  Sean A. Spence,et al.  Descartes' Error: Emotion, Reason and the Human Brain , 1995 .

[17]  Ioanna-Ourania Stathopoulou,et al.  Automated Processing and Classification of Face Images for Human–Computer Interaction Applications , 2008 .

[18]  Tsuyoshi Moriyama,et al.  Measurement of human vocal emotion using fuzzy control , 2001, Systems and Computers in Japan.

[19]  Hillary Anger Elfenbein,et al.  When familiarity breeds accuracy: cultural exposure and facial emotion recognition. , 2003, Journal of personality and social psychology.

[20]  A. Damasio.  Descartes’ Error. Emotion, Reason and the Human Brain. New York (Grosset/Putnam) 1994. , 1994 .

[21]  Zhihong Zeng,et al.  Bimodal HCI-related affect recognition , 2004, ICMI '04.

[22]  Peter C. Fishburn,et al.  Letter to the Editor - Additive Utilities with Incomplete Product Sets: Application to Priorities and Assignments , 1967, Oper. Res..

[23]  Rosalind W. Picard.  Affective computing: challenges , 2003, Int. J. Hum. Comput. Stud..