Towards an Affective Cognitive Architecture

Towards an Affective Cognitive Architecture Markus Guhe, Wayne D. Gray, Michael J. Schoelles, Quiang Ji (guhem, grayw, schoem, jiq@rpi.edu) Rensselaer Polytechnic Institute Affective state can influence users’ cognitive processing ca- pabilities and hence their productivity (Picard 1997). The first goal of our research is to develop methods to timely and efficiently recognize negative user affective states, model their influence on cognition and behavior, and pro- vide the most appropriate intervention in a timely manner to return the user to his/her productive state. The second, more distal, goal is to develop an integrated architecture of affect and cognition. There are four challenges facing this initia- tive. (1) Users’ affect develops over time, and its expres- sions vary significantly with individual and context. (2) Af- fective state observations from a given sensory source are ambiguous, uncertain, and incomplete. (3) The influence of cognition on affective state and vice versa is not well under- stood. (4) Interventions to improve user performance must be timely and effective. Our approach contrasts with the state-of-the-art in aug- mented cognition as well as in affect-based augmentation. The former assumes normative performance and fails to adapt to the user’s current affective state. The latter tends to have low-to-no cognitive fidelity, failing to understand the cognitive activities that lead to the observed user state. (But see Hudlicka 2003 for an overview of recent approaches.) Our framework addresses both sets of challenges. The proposed framework has five major parts: data sens- ing, user affective modeling, user cognitive modeling, an integrated affective–cognitive model, and a probabilistic user assistance model. Data sensing entails various visual, physiological, and behavioral data. The Rensselaer Bayesian Affect Recognition System (R- BARS) determines the user’s most likely affective states using both current and stored sensory data. The model’s context component represents information about relevant environmental factors such as time of day and type of work. The affective state component represents the affective states the system can infer. The affective state we are currently in- vestigating is confusion. The profile component may include experience, skill level, etc. It enables us to adapt the model to individual differences. Finally, the model’s observation component integrates current data with the longitudinal data record collected during the session. The Rensselaer Cognitive Architecture of Cognition (RAAC) is based on ACT-R (Anderson & Lebiere, 1998). Our current focus is on “model tracing”; i.e., the step-by- step tracing of human performance in real-time. Although model tracing in real-time has been repeatedly demonstrated at the 10s level of analysis (Anderson, 2002), behavior at the 100ms level, such as point-of-gaze, is viewed as non- deterministic. We use a dual-task to induce confusion in the user. The math task is a simple addition/subtraction of two-digit num- bers. The user must decide whether the result presented on the screen is correct. The audio task is to determine whether a letter is lower or higher in the alphabet than the previously presented one. For example, for the sequence a–c–b the user must press the higher key first and then the lower key. The tasks are presented in eight 10min blocks that are sub- divided into 36s intervals. For each task, one stimulus is presented every 2s, 4s, or 6s, so that there are 18, 9, or 6 stimuli per interval. By varying the rate of presentation be- tween intervals for each task we get 9 different combina- tions, e.g. 6–18: 6 stimuli per interval in the math task and 18 stimuli in the audio task. Varying these combinations varies the user’s level of confusion, which is confirmed by pilot data. Although performing the audio task at a rate of 2s is manageable, the math task proves very challenging – in particular in conjunction with the audio task. On trials with challenging (18–18) schedules the performance over a 10min block can drop below 20% for the math task (it is typically around 60%). The audio task is usually signifi- cantly better; even for challenging schedules the perform- ance hardly drops below 80%. The cognitive implications of the user’s affective state are established by analyzing the deviation of user behavior from the optimal path determined by the model. We will interpret the difference between expected and observed behavior as the influence of affect on cognition and behavior. In combining R-BARS with RAAC our proximal goal is to mimic the effect of affect by identifying low-level pa- rameters of the cognitive architecture that, when varied, mimic the cognitive and behavioral consequences of affec- tive state. Candidate parameters include noise in memory activation and noise in production choice, cf. Belavkin We face a profound challenge. Even developing a reliable affective–cognitive model for the task at hand is demanding. Yet, even a partially validated integrated affective–cognitive model would be an important step forward for understand- ing of the relationship between cognition and affect. Acknowledgments Research supported by a grant from Office of Naval Research. References Anderson, J. R. 2002. Spanning seven orders of magnitude: A challenge for cognitive modeling. Cognitive Science, 26(1):85- Anderson, J. R., & Lebiere, C. eds. 1998. The atomic components of thought. Hillsdale, NJ: Lawrence Erlbaum. Belavkin, R. V. 2001. Modelling the inverted-U effect with ACT-R. Proceedings of ICCM–4, Mahwah, NJ. Hudlicka, E. 2003. To feel or not to feel: The role of affect in hu- man–computer interaction. International Journal of Human- Computer Studies, 59(1):1–32. Picard, R. W. 1997. Affective computing. Cambridge: MIT Press.