An experimental assessment of semantic apprehension of graphical linguistics

Visual displays of information (such as dashboards) have become very sophisticated in rendering world semantics but neglect display semantics, leading to what is commonly called information overload. Underlying storage and retrieval research has been utilizing semantic and cognitive theory to drive the current implementations of ontology markup using the resource description framework (RDF) and Web ontology language (OWL) for over a decade. Yet despite these semantically rich underlying description logics, and despite the very large and mature stream of cognitive and neuroscience theory literature on visual perception and attention, memory, and linguistics, this is one aspect within the area of information visualization and human factors research where empirically tested semantic theory has not yet caught up, and begs for theory-driven research into display semantics using what might be termed ''graphical linguistics.'' We conducted an experiment to assess the cognitive effort of interpreting domain general knowledge using the same information represented in three forms, and found that graphical linguistics reduce cognitive effort for a specific type of task involving high-density time-sensitive information typically found in situation control rooms.

[1]  Michael F. Morris.  Kiviat graphs: conventions and "figures of merit" , 1974, PERV.

[2]  Mark H. Johnson.  The Development of Visual Attention: A Cognitive Neuroscience Perspective , 2008 .

[3]  John Sweller,et al.  Cognitive Load During Problem Solving: Effects on Learning , 1988, Cogn. Sci..

[4]  Alan Richardson-Klavehn,et al.  Level of processing and the process-dissociation procedure: Elusiveness of null effects on estimates of automatic retrieval , 2002, Memory.

[5]  Kevin B. Bennett,et al.  An Empirical Comparison of Alternative Methodologies for the Evaluation of Configural Displays , 2000, Hum. Factors.

[6]  Daniel G Bobrow,et al.  On data-limited and resource-limited processes , 1975, Cognitive Psychology.

[7]  Herman Chernoff,et al.  The Use of Faces to Represent Points in k- Dimensional Space Graphically , 1973 .

[8]  L. Reder.  Implicit memory and metacognition , 1996 .

[9]  Brian McBride,et al.  The Resource Description Framework (RDF) and its Vocabulary Description Language RDFS , 2004, Handbook on Ontologies.

[10]  Kim J. Vicente,et al.  Coping with Human Errors through System Design: Implications for Ecological Interface Design , 1989, Int. J. Man Mach. Stud..

[11]  Laurent Petit,et al.  N170 ERPs could represent a logographic processing strategy in visual word recognition , 2007, Behavioral and Brain Functions.

[12]  Susan Bell Trickett,et al.  Note-Taking for Self-Explanation and Problem Solving , 2001, Hum. Comput. Interact..

[13]  E. Bizzi,et al.  The Cognitive Neurosciences , 1996 .

[14]  Richard J. McNally,et al.  Cognitive Bias in Panic Disorder: A Process Dissociation Approach to Automaticity , 2001, Cognitive Therapy and Research.

[15]  H Kondo,et al.  A computer system applying the face method to represent multiphasic tests. , 1987, Medical informatics = Medecine et informatique.

[16]  Richard E. Mayer,et al.  Multimedia Learning , 2001, Visible Learning Guide to Student Achievement.

[17]  G. W. Furnas,et al.  Generalized fisheye views , 1986, CHI '86.

[18]  G. A. Miller.  THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[19]  J. Fleiss,et al.  Intraclass correlations: uses in assessing rater reliability. , 1979, Psychological bulletin.

[20]  James J. Thomas,et al.  Visualizing the non-visual: spatial analysis and interaction with information from text documents , 1995, Proceedings of Visualization 1995 Conference.

[21]  P. Kline,et al.  Validation of the AGARD STRES Battery of Performance Tests , 1996, Hum. Factors.

[22]  Hsinchun Chen,et al.  Automated criminal link analysis based on domain knowledge , 2007, J. Assoc. Inf. Sci. Technol..

[23]  Hideki Koike,et al.  Fractal views: a fractal-based method for controlling information display , 1995, TOIS.

[24]  Ben Shneiderman,et al.  Designing the user interface (2nd ed.): strategies for effective human-computer interaction , 1992 .

[25]  Christopher D. Wickens,et al.  The Proximity Compatibility Principle: Its Psychological Foundation and Relevance to Display Design , 1995, Hum. Factors.

[26]  George R. S. Weir,et al.  Human Computer Interaction and Complex Systems , 1991 .

[27]  Steffen Staab,et al.  Handbook on Ontologies in Information Systems , 2003 .

[28]  F. Paas,et al.  Cognitive Load Theory and Instructional Design: Recent Developments , 2003 .

[29]  Clare Dyer.  Doctors go on trial for manslaughter after removing wrong kidney , 2002, BMJ : British Medical Journal.

[30]  Jennifer Xu,et al.  Automated criminal link analysis based on domain knowledge: Research Articles , 2007 .

[31]  B. Dosher,et al.  The role of attention in the programming of saccades , 1995, Vision Research.

[32]  Jonathan C. Roberts,et al.  The Craft of Information Visualization , 2008 .

[33]  James T. Enns,et al.  High-speed visual estimation using preattentive processing , 1996, TCHI.

[34]  Erik D. Reichle,et al.  Tests of the E-Z Reader model: Exploring the interface between cognition and eye-movement control , 2006, Cognitive Psychology.

[35]  Mary Hegarty,et al.  Visuospatial working memory, central executive functioning, and psychometric visuospatial abilities: How are they related , 2001 .

[36]  Demaris A. Montgomery,et al.  Human Sensitivity to Variability Information in Detection Decisions , 1999, Hum. Factors.

[37]  Graham Cooper,et al.  Cognitive load theory as an aid for instructional design , 1990 .

[38]  J. Bain,et al.  PSYCHOLOGICAL SCIENCE Research Article How Many Variables Can Humans Process? , 2022 .

[39]  Edward R. Tufte,et al.  The Visual Display of Quantitative Information , 1986 .

[40]  Edward E. Smith,et al.  Working memory in humans: Neuropsychological evidence. , 1995 .

[41]  R. Remington,et al.  The role of input and output modality pairings in dual-task performance: Evidence for content-dependent central interference , 2006, Cognitive Psychology.

[42]  John R. Goodall,et al.  A user-centered look at glyph-based security visualization , 2005, IEEE Workshop on Visualization for Computer Security, 2005. (VizSEC 05)..

[43]  Allison L Miller,et al.  An association study of DRD2 and COMT polymorphisms with novelty seeking and harm avoidance scores, in two independent samples of depressed patients , 2007, Behavioral and Brain Functions.

[44]  John R. Anderson.  Cognitive Psychology and Its Implications , 1980 .

[45]  Herbert A. Simon,et al.  Why a Diagram is (Sometimes) Worth Ten Thousand Words , 1987 .

[46]  L. Reder,et al.  Metacognition does not imply awareness: Strategy choice is governed by implicit learning and memory , 1996 .

[47]  Kenneth W. Kolence,et al.  Software unit profiles & Kiviat figures , 1973, PERV.

[48]  Kun Chang Lee,et al.  Causal knowledge-based design of EDI controls: an explorative study , 2007, Comput. Hum. Behav..

[49]  G. Bower,et al.  THE PSYCHOLOGY OF LEARNING AND M·OTIVATION , 2001 .

[50]  R. Sternberg,et al.  Intelligence, Information Processing and Analogical Reasoning : The Componential Analysis of Human Abilities , 1977 .

[51]  M F Lesser.  GIFIC. A graphical interface for intensive care. , 1994, Proceedings. Symposium on Computer Applications in Medical Care.

[52]  R. Kozma.  Learning with Media , 1991 .

[53]  T. M. Barnhardt,et al.  Number of solutions effects in stem decision: Support for the distinction between identification and production processes in priming , 2005, Memory.

[54]  Vincent Bergeron.  Anatomical and Functional Modularity in Cognitive Science: Shifting the Focus , 2007 .

[55]  John D. Bransford,et al.  The abstraction of linguistic ideas , 1971 .

[56]  M. Posner.  Chronometric explorations of mind : the third Paul M. Fitts lectures, delivered at the University of Michigan, September 1976 , 1978 .

[57]  Mica R. Endsley,et al.  Designing for Situation Awareness : An Approach to User-Centered Design , 2003 .

[58]  Aaron B. Hoffman,et al.  Eyetracking and selective attention in category learning , 2005, Cognitive Psychology.

[59]  B. Schneiderman,et al.  Designing the User Interface. Strategies for Effective Human-Computer Interaction , 1992 .

[60]  Ben Shneiderman,et al.  Readings in information visualization - using vision to think , 1999 .

[61]  D. Leutner,et al.  Assessment of cognitive load in multimedia learning using dual-task methodology. , 2002, Experimental psychology.

[62]  N. Cowan.  The magical number 4 in short-term memory: A reconsideration of mental storage capacity , 2001, Behavioral and Brain Sciences.

[63]  K B Bennett,et al.  Graphical Displays: Implications for Divided Attention, Focused Attention, and Problem Solving , 1992, Human factors.

[64]  S J McKelvie,et al.  Does memory contaminate test-retest reliability? , 1992, The Journal of general psychology.

[65]  G. Cruccu,et al.  Star-like display of EEG spectral values. , 1980, Electroencephalography and clinical neurophysiology.

[66]  Nicole B. Koppel,et al.  So Much Information, So Little Time: Evaluating Web Resources with Search Engines , 2002 .

[67]  Ben Shneiderman,et al.  Ordered and quantum treemaps: Making effective use of 2D space to display hierarchies , 2002, TOGS.

[68]  T. A. Gavrilova,et al.  The cognitive approach to the creation of ontology , 2007, Automatic Documentation and Mathematical Linguistics.

[69]  Lynn C. Robertson,et al.  The neurology of visual attention. , 1995 .

[70]  John Duncan,et al.  Effect of Template Complexity on Visual Search and Dual-Task Performance , 2005, Psychological science.

[71]  S. Langer.  Philosophy in a new key : a study in the symbolism of reason, rite, and art , 1944 .

[72]  Michael J. Albers,et al.  Information design considerations for improving situation awareness in complex problem-solving , 1999, SIGDOC '99.

[73]  Owen Dyer.  Clinical director admits patient care was “compromised” , 2002, BMJ : British Medical Journal.

[74]  Ravin Balakrishnan,et al.  Temporal Thumbnails: rapid visualization of time-based viewing data , 2004, AVI.

[75]  A. Marcus.  Dashboards in your future , 2006, INTR.

[76]  C. Spielberger,et al.  Stress and Anxiety , 1981 .

[77]  R. Desimone,et al.  Multiple memory systems in the visual cortex. , 1995 .

[78]  Michael Workman,et al.  An exploratory study of cognitive load in diagnosing patient conditions. , 2007, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[79]  L. Jacoby.  A process dissociation framework: Separating automatic from intentional uses of memory , 1991 .

[80]  Noam Chomsky.  Human Language and Other Semiotic Systems , 1979 .

[81]  Walter Schneider,et al.  Controlled and automatic human information processing: II. Perceptual learning, automatic attending and a general theory. , 1977 .

[82]  T. Chartrand,et al.  THE UNBEARABLE AUTOMATICITY OF BEING , 1999 .

[83]  G E Legge,et al.  Efficiency of graphical perception , 1991, Perception & psychophysics.

[84]  L. James.  Aggregation Bias in Estimates of Perceptual Agreement. , 1982 .

[85]  Stephen B. Johnson,et al.  Review: An Object-oriented Taxonomy of Medical Data Presentations , 2000, J. Am. Medical Informatics Assoc..

[86]  M. Bar,et al.  Top-down predictions in the cognitive brain , 2007, Brain and Cognition.

[87]  R N Fleischman,et al.  Modeling the Cognitive Content of Displays , 1989, Human factors.

[88]  Shayne Loft,et al.  Modeling and Predicting Mental Workload in En Route Air Traffic Control: Critical Review and Broader Implications , 2007, Hum. Factors.