Wearable and automotive systems for affect recognition from physiology

Novel systems and algorithms have been designed and built to recognize affective patterns in physiological signals. Experiments were conducted for evaluation of the new systems and algorithms in three types of settings: a highly constrained laboratory setting, a largely unconstrained ambulatory environment, and a less unconstrained automotive environment. The laboratory experiment was designed to test for the presence of unique physiological patterns given a relatively motionless seated subject, intentionally expressing one of eight different emotions. This experiment generated a large dataset of physiological signals containing many day-today variations, and the proposed features contributed to a success rate of 81% for discriminating all eight emotions and up to 100% for subsets of emotion based on similar emotion qualities. For experiments in a largely unconstrained ambulatory setting, new wearable computer systems and sensors were developed and tested on subjects who walked, jogged, talked, and otherwise went about daily activities. Although physical motion often overwhelmed affective signals in this context, the systems developed in this thesis are currently useful as activity monitors, providing an image diary correlated with physiological signals. A third experiment was conducted in the natural but physically constrained environment of an automobile, generating a large database of physiological signals covering over 40 hours of driving data. This experiment was designed to induce varying amounts of stress in three conditions: rest, city diving and highway driving. Algorithms for detecting driver stress achieved a recognition rates of 96% for stress ratings based on task conditions and 89% accuracy using ratings of perceived stress from driver questionnaires. The results of second by second video coding of stressors by independent coders showed highly significant correlations with physiological features (up to r = .77 for over 4000 samples). Together, these three experiments show a range of success in recognizing affect from physiology, highlighting the need for more automatic context sensing in unconstrained conditions, and showing high recognition rates given somewhat constrained conditions. The recognition rates obtained thus far lend support to the hypothesis that many emotional differences can be automatically discriminated in patterns of physiological changes. (Copies available exclusively from MIT Libraries, Rm. 14-0551, Cambridge, MA 02139-4307. Ph. 617-253-5668; Fax 617-253-1690.)