HMM Modeling of User Mood through Recognition of Vocal Emotions

This paper aims at defining a real-time probabilistic model for user’s mood in its dialect with a software agent, which has a long-term goal of counseling the user in the domain of “coping with exam pressure”. We propose a new approach based on Hidden Markov Models (HMMs) to describe the differences in the sequence of emotions expressed due to different moods experienced by users. During real time operation, each user move is passed on to a vocal affect recognizer. The decisions from the recognizer about the kind of emotion expressed are then mapped into code-words to generate a sequence of discrete symbols for HMM models of each mood. We train and test the system using corpora of the temporal sequences of tagged emotional utterances by six male and six female adult Indians in English and Hindi language. Our system achieved an average f-measure rating for all moods of approximately 78.33%.

[1]  M. Bartlett,et al.  Machine Analysis of Facial Expressions , 2007 .

[2]  Eugene Charniak,et al.  Statistical language learning , 1997 .

[3]  Yuri Ivanov,et al.  Probabilistic combination of multiple modalities to detect interest , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[4]  P. Schnurr,et al.  Mood: The Frame of Mind , 2011 .

[5]  H. Niemann,et al.  Private Emotions vs. Social Interaction-towards New Dimensions in Research on Emotion , 2005 .

[6]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[7]  Ashish Kapoor,et al.  Multimodal affect recognition in learning environments , 2005, ACM Multimedia.

[8]  Roberto Pieraccini,et al.  Using Markov decision process for learning dialogue strategies , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[9]  Björn W. Schuller,et al.  Hidden Markov model-based speech emotion recognition , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[10]  Nicole Novielli,et al.  HMM modeling of user engagement in advice-giving dialogues , 2010, Journal on Multimodal User Interfaces.

[11]  Ankit Agrawal,et al.  Recognition of vocal emotions from acoustic profile , 2012, ICACCI '12.

[12]  Rosalind W. Picard.  Affective computing: challenges , 2003, Int. J. Hum. Comput. Stud..

[13]  Andreas Wendemuth,et al.  TUNING HIDDEN MARKOV MODEL FOR SPEECH EMOTION RECOGNITION , 2007 .

[14]  Anthony Fox.  Prosodic Features and Prosodic Structure: The Phonology of Suprasegmentals , 2000 .

[15]  Andreas Stolcke,et al.  Dialogue act modeling for automatic tagging and recognition of conversational speech , 2000, CL.

[16]  Larry S. Davis,et al.  Learning dynamics for exemplar-based gesture recognition , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..