Using fNIRS brain sensing to evaluate information visualization interfaces

We show how brain sensing can lend insight to the evaluation of visual interfaces and establish a role for fNIRS in visualization. Research suggests that the evaluation of visual design benefits by going beyond performance measures or questionnaires to measurements of the user's cognitive state. Unfortunately, objectively and unobtrusively monitoring the brain is difficult. While functional near-infrared spectroscopy (fNIRS) has emerged as a practical brain sensing technology in HCI, visual tasks often rely on the brain's quick, massively parallel visual system, which may be inaccessible to this measurement. It is unknown whether fNIRS can distinguish differences in cognitive state that derive from visual design alone. In this paper, we use the classic comparison of bar graphs and pie charts to test the viability of fNIRS for measuring the impact of a visual design on the brain. Our results demonstrate that we can indeed measure this impact, and furthermore measurements indicate that there are not universal differences in bar graphs and pie charts.

[1]  J. G. Hollands,et al.  Engineering Psychology and Human Performance , 1984 .

[2]  Fred G. W. C. Paas,et al.  The Efficiency of Instructional Conditions: An Approach to Combine Mental Effort and Performance Measures , 1992 .

[3]  A. Blasi,et al.  Illuminating the developing brain: The past, present and future of functional near infrared spectroscopy , 2010, Neuroscience & Biobehavioral Reviews.

[4]  F. Paas,et al.  Cognitive Load Measurement as a Means to Advance Cognitive Load Theory , 2003 .

[5]  E Zarahn,et al.  Event-related functional MRI: implications for cognitive psychology. , 1999, Psychological bulletin.

[6]  A. Owen,et al.  Anterior prefrontal cortex: insights into function from anatomy and neuroimaging , 2004, Nature Reviews Neuroscience.

[7]  R. Passingham,et al.  The prefrontal cortex: response selection or maintenance within working memory? , 2000, 5th IEEE EMBS International Summer School on Biomedical Imaging, 2002..

[8]  Giuseppe Santucci,et al.  BELIV'08: Beyond time and errors: novel evaluation methods for information visualization , 2008, CHI Extended Abstracts.

[9]  Robert J. K. Jacob,et al.  Using fNIRS brain sensing in realistic HCI settings: experiments and guidelines , 2009, UIST '09.

[10]  Kathryn M. McMillan,et al.  N‐back working memory paradigm: A meta‐analysis of normative functional neuroimaging studies , 2005, Human brain mapping.

[11]  Ronald N. Goodman,et al.  Neural decoding of treadmill walking from noninvasive electroencephalographic signals. , 2011, Journal of neurophysiology.

[12]  I. Spence Visual psychophysics of simple graphical elements. , 1990, Journal of experimental psychology. Human perception and performance.

[13]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[14]  Masaaki Kurosu,et al.  Human Centered Design, First International Conference, HCD 2009, Held as Part of HCI International 2009, San Diego, CA, USA, July 19-24, 2009, Proceedings , 2009, HCI.

[15]  Matthias Scheutz,et al.  Sensing cognitive multitasking for a brain-based adaptive user interface , 2011, CHI.

[16]  Jeffrey Heer,et al.  Crowdsourcing graphical perception: using mechanical turk to assess visualization design , 2010, CHI.

[17]  Ian Spence,et al.  No Humble Pie: The Origins and Usage of a Statistical Chart , 2005 .

[18]  David A. Boas,et al.  A Quantitative Comparison of Simultaneous BOLD fMRI and NIRS Recordings during Functional Brain Activation , 2002, NeuroImage.

[19]  Robert J. K. Jacob,et al.  Distinguishing Difficulty Levels with Non-invasive Brain Activity Measurements , 2009, INTERACT.

[20]  Matthias Scheutz,et al.  Brainput: enhancing interactive systems with streaming fnirs brain input , 2012, CHI.

[21]  Dick de Waard,et al.  The measurement of drivers' mental workload , 1996 .

[22]  Cláudio T. Silva,et al.  A User Study of Visualization Effectiveness Using EEG and Cognitive Load , 2011, Comput. Graph. Forum.

[23]  B. Chance,et al.  A novel method for fast imaging of brain function, non-invasively, with light. , 1998, Optics express.

[24]  William F. Moroney,et al.  A comparison of two scoring procedures with the NASA task load index in a simulated flight task , 1992, Proceedings of the IEEE 1992 National Aerospace and Electronics Conference@m_NAECON 1992.

[25]  Edward Cutrell,et al.  "Yours is better!": participant response bias in HCI , 2012, CHI.

[26]  Stephen H. Fairclough,et al.  Fundamentals of physiological computing , 2009, Interact. Comput..

[27]  Hasan Ayaz,et al.  Optical brain monitoring for operator training and mental workload assessment , 2012, NeuroImage.

[28]  Robert J. K. Jacob,et al.  This is your brain on interfaces: enhancing usability testing with functional near-infrared spectroscopy , 2011, CHI.

[29]  Martin Wolf,et al.  Progress of near-infrared spectroscopy and topography for brain and muscle clinical applications. , 2007, Journal of biomedical optics.

[30]  Eytan Adar,et al.  Benefitting InfoVis with Visual Difficulties , 2011, IEEE Transactions on Visualization and Computer Graphics.

[31]  S. Lewandowsky,et al.  Displaying proportions and percentages , 1991 .

[32]  C. Wickens,et al.  Dissociation of Performance and Subjective Measures of Workload , 1988 .

[33]  Weidong Huang,et al.  Measuring Effectiveness of Graph Visualizations: A Cognitive Load Perspective , 2009, Inf. Vis..

[34]  David K. Simkin,et al.  An Information-Processing Analysis of Graph Perception , 1987 .

[35]  Human-Computer Interaction – INTERACT 2009 , 2009, Lecture Notes in Computer Science.

[36]  Robert J. K. Jacob,et al.  Brain measurement for usability testing and adaptive interfaces: an example of uncovering syntactic workload with functional near infrared spectroscopy , 2009, CHI.

[37]  Robert J. K. Jacob,et al.  From Brain Signals to Adaptive Interfaces: Using fNIRS in HCI , 2010, Brain-Computer Interfaces.

[38]  A. Villringer,et al.  Non-invasive optical spectroscopy and imaging of human brain function , 1997, Trends in Neurosciences.

[39]  P. Hancock,et al.  Human Mental Workload , 1988 .

[40]  Christophe Hurter,et al.  The Physiological User's Response as a Clue to Assess Visual Variables Effectiveness , 2009, HCI.

[41]  W. Cleveland,et al.  Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods , 1984 .

[42]  Daniel J. Wigdor,et al.  Perception of elementary graphical elements in tabletop and multi-surface environments , 2007, CHI.