Emotion research and AI: Some theoretical and technical issues*

This paper describes an approach that uses interactive computer games as an experimental setting for the study of emotion, behavior, and nonverbal communication. In the first, more technical part, we present a method that allows to automatically code facial behavior of a subject playing a computer game. In this procedure, facial behavior is registered automatically with the aid of small plastic dots which are affixed to pre-defined regions of the subjects face. The resulting dot patterns are then classified with an artificial neural network. In the second part of the article, we discuss the theoretical framework of our experimental approach. In this view, the computer game provides a relatively simple but complete context a microworld in the sense of Toda’s 'Fungus Eater' for the interpretation of the internal emotional and cognitive regulatory processes. In the third part, we contrast the empirical work with a synthetic approach where we can use the theoretical assumptions and the empirical data to model an autonomous agent as an emotional problemsolver.