An EEG-based Approach for Evaluating Graphic Icons from the Perspective of Semantic Distance

Graphic icons play an increasingly important role in interface design due to the proliferation of digital devices in recent years. Their ability to express information in a universal fashion allows us to immediately interact with new applications, systems, and devices. Icons can, however, cause user confusion and frustration if designed poorly. Several studies have evaluated icons using behavioral-performance metrics such as reaction time as well as self-report methods. However, determining the usability of icons based on behavioral measures alone is not straightforward, because users' interpretations of the meaning of icons involve various cognitive processes and perceptual mechanisms. Moreover, these perceptual mechanisms are affected not only by the icons themselves, but by usage scenarios. Thus, we need a means of sensitively and continuously measuring users' different cognitive processes when they are interacting with icons. In this study, we propose an EEG-based approach to icon evaluation, in which users' EEG signals are measured in multiple usage scenarios. Based on a combination of EEG and behavioral results, we provide a novel interpretation of the participants' perception during these tasks, and identify some important implications for icon design.

[1]  Thorsten O. Zander,et al.  Enhancing Human-Computer Interaction with Input from Active and Passive Brain-Computer Interfaces , 2010, Brain-Computer Interfaces.

[2]  Siné McDougall,et al.  What’s in a name? The role of graphics, functions, and their interrelationships in icon identification , 2009, Behavior research methods.

[3]  Seana Coulson,et al.  Iconic gestures prime related concepts: An ERP study , 2007, Psychonomic bulletin & review.

[4]  Terrence J. Sejnowski,et al.  Independent Component Analysis of Simulated ERP Data , 2000 .

[5]  Chi Thanh Vi,et al.  Error related negativity in observing interactive tasks , 2014, CHI.

[6]  K. Lowenberg,et al.  Cultural differences in sensitivity to social context: Detecting affective incongruity using the N400 , 2013, Social neuroscience.

[7]  Robert J. K. Jacob,et al.  Using fNIRS brain sensing to evaluate information visualization interfaces , 2013, CHI.

[8]  Rock Leung Improving the learnability of mobile device applications for older adults , 2009, CHI Extended Abstracts.

[9]  Vidya Setlur,et al.  Semanticons: Visual Metaphors as File Icons , 2005, Comput. Graph. Forum.

[10]  Edward Cutrell,et al.  "Yours is better!": participant response bias in HCI , 2012, CHI.

[11]  Martin B. Curry,et al.  Icon Identification in Context: The Changing Role of Icon Characteristics With User Experience , 2007, Hum. Factors.

[12]  Xiaofu Tang,et al.  Event-related potential N270 is elicited by mental conflict processing in human brain , 2000, Neuroscience Letters.

[13]  Jonathan R. Folstein,et al.  Novelty and conflict in the categorization of complex stimuli. , 2008, Psychophysiology.

[14]  Carryl L. Baldwin,et al.  The Role of Age-Related Neural Timing Variability in Speech Processing , 2011 .

[15]  Stephen A. Brewster,et al.  Multiple notification modalities and older users , 2013, CHI.

[16]  Timothy C. Clausner,et al.  Perceptual Cues and Imagined Viewpoints Modulate Visual Search in Air Traffic Control Displays , 2009 .

[17]  Peter Charles Woods,et al.  Experimental Color in Computer Icons , 2009, VINCI.

[18]  Randolph G. Bias,et al.  How are icons processed by the brain? Neuroimaging measures of four types of visual stimuli used in information systems , 2015, J. Assoc. Inf. Sci. Technol..

[19]  Kara D. Federmeier,et al.  Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP). , 2011, Annual review of psychology.

[20]  Yang-Kun Ou,et al.  Effects of sign design features and training on comprehension of traffic signs in Taiwanese and Vietnamese user groups , 2012 .

[21]  Chris Harrison,et al.  Kineticons: using iconographic motion in graphical user interface design , 2011, CHI.

[22]  Ranjana K. Mehta,et al.  Neuroergonomics: a review of applications to physical and cognitive work , 2013, Front. Hum. Neurosci..

[23]  Kerem Rizvanoglu,et al.  Cross-Cultural Understanding of the Dual Structure of Metaphorical Icons: An Explorative Study with French and Turkish Users on an E-Learning Site , 2009, HCI.

[24]  R. Yerkes,et al.  The relation of strength of stimulus to rapidity of habit‐formation , 1908 .

[25]  S. Goto,et al.  Cultural differences in the visual processing of meaning: detecting incongruities between background and foreground objects using the N400. , 2010, Social cognitive and affective neuroscience.

[26]  J. Ward,et al.  The Student's Guide to Cognitive Neuroscience , 2006 .

[27]  Jonathan Grainger,et al.  Words and pictures: An electrophysiological investigation of domain specific processing in native Chinese and English speakers , 2011, Neuropsychologia.

[28]  Ivan Burmistrov,et al.  Flat Design vs Traditional Design: Comparative Experimental Study , 2015, INTERACT.

[29]  Chi Thanh Vi,et al.  Detecting error-related negativity for interaction design , 2012, CHI.

[30]  Jonathan R. Folstein,et al.  Influence of cognitive control and mismatch on the N2 component of the ERP: a review. , 2007, Psychophysiology.

[31]  Li-Wei Ko,et al.  An EEG-based approach for evaluating audio notifications under ambient sounds , 2014, CHI.

[32]  Chia-lin Lee,et al.  Effects of semantic constraint and cloze probability on Chinese classifier-noun agreement , 2014, Journal of Neurolinguistics.

[33]  Antonio Krüger,et al.  A study on icon arrangement by smartphone users , 2013, CHI.

[34]  Ashweeni Kumar Beeharee,et al.  The case of the missed icon: change blindness on mobile devices , 2012, CHI.

[35]  J. Polich Updating P300: An integrative theory of P3a and P3b , 2007, Clinical Neurophysiology.

[36]  Qingguo Ma,et al.  Event-related potential N270 correlates of brand extension , 2007, Neuroreport.

[37]  Daniel Afergan,et al.  Phylter: A System for Modulating Notifications in Wearables Using Physiological Sensing , 2015, HCI.

[38]  Martin Porcheron,et al.  Measuring the effect of think aloud protocols on workload using fNIRS , 2014, CHI.

[39]  Martin B. Curry,et al.  Measuring symbol and icon characteristics: Norms for concreteness, complexity, meaningfulness, familiarity, and semantic distance for 239 symbols , 1999, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[40]  T. Ethofer,et al.  His or mine? The time course of self–other discrimination in emotion processing , 2011, Social neuroscience.

[41]  Der-Song Lee,et al.  Color combination and exposure time on legibility and EEG response of icon presented on visual display terminal , 2013, Displays.

[42]  Daniel Afergan,et al.  Dynamic difficulty using brain metrics of workload , 2014, CHI.

[43]  Peter Graf,et al.  Age-related differences in the initial usability of mobile device icons , 2011, Behav. Inf. Technol..

[44]  Glyn W. Humphreys,et al.  From objects to names: A cognitive neuroscience approach , 1999, Psychological research.

[45]  Werner Sommer,et al.  The sacred and the absurd––an electrophysiological study of counterintuitive ideas (at sentence level) , 2012, Social neuroscience.

[46]  Andreas Bulling,et al.  Cognition-Aware Computing , 2014, IEEE Pervasive Computing.

[47]  Vidya Setlur,et al.  Automatic generation of semantic icon encodings for visualizations , 2014, CHI.