Probabilistic Multisensory Emotion Estimation Framework for Assistive Robotic Applications

Computer-based emotion recognition is an emerging field with envisioned applications ranging from customer satisfaction evaluation to human-machine interaction. In this paper we present a general framework for continuous emotion inference based on Bayesian biometric data fusion and the circumplex model of affect. We apply this framework to the field of assistive robotics focused on elderly and impaired people who require a wheelchair for mobility purposes. The objective is to provide an emotion-based safety layer that complements the classical collision avoidance approaches typically included in these systems. In many real-case applications the calculation of the emotional valence is not feasible, and therefore we also present here a promising novel context-based alternative currently under development. Introduction Humans seem most capable (probably due to self introspection) of distinguishing certain emotional states in other individuals just by glancing at them for a short period of time, extracting behavioural, facial and other relevant cues. Computers on the other hand, lack such capabilities and therefore need to be trained for that purpose. Due to the difference between the perceptual and analytical abilities of humans and computer, however, cues that are most suitable for us (e.g. gestures, poses, expressions) may result significantly less useful for a computer system. On the other hand, some other information of a more ”numerical nature” (e.g. electrocardiogram measurements), not that indicative for humans, could provide a computer system with key insights into the emotional state of an individual. The work we present here is the result of our preliminary research into continuous emotion estimation and its application to the development of an emotion-based navigation assistive layer for an intelligent wheelchair typically operated by a physically and/or cognitively impaired elderly person (our target user). This layer is completely user-centric and provides a technology that the user utilises unconsciously. Its task is to adapt the level of support that the user receives from the wheelchair based on the physical and cognitive capabilities of the patient, his/her driving performance, the navigation context and the user’s biometric readings. Copyright c © 2008, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. Our work differs from those of other authors in the field in a number of aspects. Firstly, it is concerned with real-time estimation of emotions, and as such, it presents techniques and mathematical models that can operate online. Secondly, while most of the related works are concerned with user satisfaction and digital entertainment, our goal is to develop robotic systems capable of reacting and adapting their behaviour in real-time (e.g. modify the navigation strategy of a mobility platform) based on the user emotions. Thirdly, due to the nature of our application and target population, we are restricted in the type and number of sensors we can use. Finally, we propose the use of context-based information in order to solve the problem of valence calculation without using intrusive or uncomfortable sensors.