SCOUT and affective interaction design: Evaluating physiological signals for usability in emotional processing

The issue of study in emotion or affect has been recently examined by Human-computer Interaction (HCI) research groups, in particular for the development of affective interaction and design. With the recent technological advances, humans are able to interact with computers in ways which are almost impossible. The new modalities for computer interaction with human emotion such skin conductivity, heart rate, brain signals and physiological signals are emerging, which extend beyond the confines of the keyboard and mouse. As such, emotion plays an important role in human-to-human communication and interaction, which allows people to express it beyond the verbal domain. The physiological signals, which are the focus of this study, play a significant method for usability study in HCI. Numbers of usability psychometrics are developed, however, a few yield “natural” interpretations, such as feelings. The physiological signals can be interpreted as a trustworthy of emotional reveal. In this study, we conduct an affective interaction and design development for an e-Learning storyboard tool called SCOUT. The paper describes the significant role of physiological signals as a method to study usability in emotion in the area of HCI and Affective Computing. Affective design and interaction are also discussed in the SCOUT development. The comparison of different research areas in HCI, focusing on different key attributes and physiological signals are studied. The result of the analysis revealed that physiological signal should be used as important as other usability psychometric in HCI, to enrich the evaluation of human-computer interaction study with human emotion.

[1]  Michael Minge,et al.  Measuring multiple components of emotions in interactive contexts , 2006, CHI Extended Abstracts.

[2]  John A. Robinson,et al.  Techniques for automated reverse storyboarding , 2005 .

[3]  Tom Carey,et al.  ACM SIGCHI Curricula for Human-Computer Interaction , 1992 .

[4]  J. Anttonen.  Using the EMFi chair to measure the user’s emotion-related heart rate responses , 2005 .

[5]  S. Salim.  Ethics And Information Privacy In Affective Computing , 2008 .

[6]  Lesley Axelrod,et al.  The affective connection: how and when users communicate emotion , 2004, CHI EA '04.

[7]  Jennifer Allanson.  Electrophysiologically Interactive Computer Systems , 2002, Computer.

[8]  D. M. Hutton,et al.  The Essence of Human Computer Interaction , 2000 .

[9]  Leysia Palen,et al.  Don't Get Emotional , 2008, Affect and Emotion in Human-Computer Interaction.

[10]  Paul Dourish,et al.  Affect: from information to interaction , 2005, Critical Computing.

[11]  Ian Jones,et al.  Storyboarding: A method for bootstrapping the design of computer-based educational tasks , 2008, Comput. Educ..

[12]  J. Lagopoulos.  Electrodermal activity , 2007, Acta Neuropsychiatrica.

[13]  D. Goleman.  Emotional Intelligence. New York (Bantam) 1995. , 1995 .

[14]  Donald A. Norman,et al.  Emotion & design: attractive things work better , 2002, INTR.

[15]  Banu Onaral,et al.  Biomedical Signals: Origin and Dynamic Characteristics; Frequency-Domain Analysis , 1999 .

[16]  Peter C. Wright,et al.  Empathy and experience in HCI , 2008, CHI.

[17]  Michael J. Lyons,et al.  Coding facial expressions with Gabor wavelets , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[18]  Robert D. Ward,et al.  Affective computing: problems, reactions and intentions , 2004, Interact. Comput..

[19]  Russell Beale,et al.  The Role of Affect and Emotion in HCI , 2008, Affect and Emotion in Human-Computer Interaction.

[20]  Peter C. Wright,et al.  The experience of enchantment in human–computer interaction , 2006, Personal and Ubiquitous Computing.

[21]  Tieniu Tan,et al.  Affective Computing: A Review , 2005, ACII.

[22]  Stephen J. Andriole.  Storyboard prototyping: a new approach to user requirements analysis , 1989 .

[23]  J. Bachorowski,et al.  Emotion in Speech , 2009 .

[24]  Valery A. Petrushin,et al.  Emotion recognition in speech signal: experimental study, development, and application , 2000, INTERSPEECH.

[25]  G Pfurtscheller,et al.  Hidden Markov Models Used for the Offline Classification of EEG Data - Hidden Markov-Modelle, verwendet zur Offline-Klassifikation von EEG-Daten , 1999, Biomedizinische Technik. Biomedical engineering.

[26]  Jun Park,et al.  AR storyboard: an augmented reality based interactive storyboard authoring tool , 2005, Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'05).

[27]  Frank Dellaert,et al.  Recognizing emotion in speech , 1996, Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96.

[28]  Johannes Wagner,et al.  From Physiological Signals to Emotions: Implementing and Comparing Selected Methods for Feature Extraction and Classification , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[29]  Roddy Cowie,et al.  FEELTRACE: an instrument for recording perceived emotion in real time , 2000 .

[30]  Anker Helms Jørgensen,et al.  Understanding affective design in a late-modernity perspective , 2003, DPPI '03.

[31]  Albrecht Schmidt,et al.  The drift table: designing for ludic engagement , 2004, CHI EA '04.

[32]  W. Buxton.  Human-Computer Interaction , 1988, Springer Berlin Heidelberg.

[33]  Han-Wei Shen,et al.  Interactive Storyboard for Overall Time-Varying Data Visualization , 2008, 2008 IEEE Pacific Visualization Symposium.

[34]  John L. Sibert,et al.  Heart rate variability: indicator of user state as an aid to human-computer interaction , 1998, CHI.

[35]  Fumio Hara,et al.  Recognition of Six basic facial expression and their strength by neural network , 1992, [1992] Proceedings IEEE International Workshop on Robot and Human Communication.

[36]  Christine Faulkner,et al.  The Essence of Human-Computer Interaction , 1997 .

[37]  K. J. Vicente,et al.  Spectral Analysis of Sinus Arrhythmia: A Measure of Mental Effort , 1987, Human factors.

[38]  Na Li,et al.  The importance of affective quality , 2005, CACM.

[39]  Robert D. Ward,et al.  Physiological responses to different WEB page designs , 2003, Int. J. Hum. Comput. Stud..

[40]  J. Lyons.  Coding facial expression with Gabor wavelets , 1998 .

[41]  Michael Minge,et al.  Consideration of Multiple Components of Emotions in Human-Technology Interaction , 2008, Affect and Emotion in Human-Computer Interaction.

[42]  Gregory D. Abowd,et al.  Human-computer interaction (2nd ed.) , 1998 .