Linking unstructured evidence to structured observations

Many professionals, like journalists, writers, or consultants, need to acquire information from various sources, make sense of this unstructured evidence, structure their observations, and finally create and deliver their product, such as a report or a presentation. In formative interviews, we found that tools allowing structuring of observations are often disconnected from the corresponding evidence. Therefore, we designed a sensemaking environment with a flexible observation graph that visually ties together evidence in unstructured documents with the user’s structured knowledge. This is achieved through bi-directional deep links between highlighted document portions and nodes in the observation graph. In a controlled study, we compared users’ sensemaking strategies using either the observation graph or a simple text editor on a large display. Results show that the observation graph represents a holistic, compact representation of users’ observations, which can be linked to unstructured evidence on demand. In contrast, users taking textual notes required much more display space to spatially organize source documents containing unstructured evidence. This implies that spatial organization is a powerful strategy to structure observations even if the available space is limited.

[1]  Jeffrey Heer,et al.  D³ Data-Driven Documents , 2011, IEEE Transactions on Visualization and Computer Graphics.

[2]  HeerJeffrey,et al.  D3 Data-Driven Documents , 2011 .

[3]  Alison Kidd,et al.  The marks are on the knowledge worker , 1994, CHI '94.

[4]  Olusola O. Adesope,et al.  Learning With Concept and Knowledge Maps: A Meta-Analysis , 2006 .

[5]  Jöran Beel,et al.  An exploratory analysis of mind maps , 2011, DocEng '11.

[6]  David Kirsh,et al.  The Intelligent Use of Space , 1995, Artif. Intell..

[7]  I. Kinchin,et al.  How a qualitative approach to concept map analysis can be used to aid learning by illustrating patterns of conceptual development , 2000 .

[8]  Dieter Schmalstieg,et al.  Show me the invisible: visualizing hidden content , 2014, CHI.

[9]  Joseph D. Novak,et al.  Theoretical Origins of Concept Maps, How to Construct Them, and Uses in Education , 2007 .

[10]  Jian Zhao,et al.  InkPlanner: Supporting Prewriting via Intelligent Visual Diagramming , 2019, IEEE Transactions on Visualization and Computer Graphics.

[11]  Morten Fjeld,et al.  RAMPARTS: Supporting Sensemaking with Spatially-Aware Mobile Interactions , 2016, CHI.

[12]  Chris North,et al.  The semantics of clustering: analysis of user-generated spatializations of text documents , 2012, AVI.

[13]  David Gotz,et al.  The ScratchPad: sensemaking support for the web , 2007, WWW '07.

[14]  Mario J. López,et al.  Study Activities That Foster Generative Learning: Notetaking, Graphic Organizer, and Questioning , 2019, Journal of Educational Computing Research.

[15]  Thomas W. Malone,et al.  How do people organize their desks?: Implications for the design of office information systems , 1983, TOIS.

[16]  Petra Isenberg,et al.  Collaborative Brushing and Linking for Co‐located Visual Analytics of Document Collections , 2009, Comput. Graph. Forum.

[17]  Keiichi Kobayashi,et al.  What limits the encoding effect of note-taking? A meta-analytic examination , 2005 .

[18]  Eser Kandogan,et al.  How a freeform spatial interface supports simple problem solving tasks , 2011, CHI.

[19]  Kate Herd,et al.  SenseMap: Supporting browser-based online sensemaking through analytic provenance , 2016, 2016 IEEE Conference on Visual Analytics Science and Technology (VAST).

[20]  Xiaojun Bi,et al.  Comparing usage of a large high-resolution display to single or dual desktop displays for daily work , 2009, CHI.

[21]  Dagobert Soergel,et al.  Process patterns and conceptual changes in knowledge representations during information seeking and sensemaking: A qualitative user study , 2016, J. Inf. Sci..

[22]  Michael S. Bernstein,et al.  Information ! Scraps : ! ! How ! and ! Why ! Information ! Eludes ! our ! Personal ! Information ! Management , 2022 .

[23]  John T. Stasko,et al.  Revisiting Display Space Management: Understanding Current Practice to Inform Next-generation Design , 2004, Graphics Interface.

[24]  Alexander W. Skaburskis,et al.  The Sandbox for analysis: concepts and methods , 2006, CHI.

[25]  Eytan Adar,et al.  texSketch: Active Diagramming through Pen-and-Ink Annotations , 2020, CHI.

[26]  Martin Davies Concept mapping, mind mapping and argument mapping: what are the differences and do they matter? , 2011 .

[27]  Melanie Tory,et al.  Supporting Communication and Coordination in Collaborative Sensemaking , 2014, IEEE Transactions on Visualization and Computer Graphics.

[28]  Chris North,et al.  Beyond visual acuity: the perceptual scalability of information visualizations for large displays , 2007, CHI.

[29]  Mario J. López,et al.  When two computer-supported learning strategies are better than one: An eye-tracking study , 2018, Comput. Educ..

[30]  Dieter Schmalstieg,et al.  Collaborative information linking: Bridging knowledge gaps between users by linking across applications , 2011, 2011 IEEE Pacific Visualization Symposium.

[31]  Analyst's Workspace: An embodied sensemaking environment for large, high-resolution displays , 2012, 2012 IEEE Conference on Visual Analytics Science and Technology (VAST).

[32]  Dieter Schmalstieg,et al.  How Sensemaking Tools Influence Display Space Usage , 2017, EuroVA@EuroVis.

[33]  Jonathan Grudin,et al.  Partitioning digital worlds: focal and peripheral awareness in multiple monitor use , 2001, CHI.

[34]  Yvonne Rogers,et al.  External cognition: how do graphical representations work? , 1996, Int. J. Hum. Comput. Stud..

[35]  Chris North,et al.  VisPorter: facilitating information sharing for collaborative sensemaking on multiple displays , 2014, Personal and Ubiquitous Computing.

[36]  Wendy E. Mackay,et al.  Effects of display size and navigation type on a classification task , 2014, CHI.

[37]  Ravin Balakrishnan,et al.  A study of tabbed browsing among mozilla firefox users , 2010, CHI.

[38]  Christopher Andrews,et al.  Space to think: large high-resolution displays for sensemaking , 2010, CHI.

[39]  Martin J. Eppler A Comparison between Concept Maps, Mind Maps, Conceptual Diagrams, and Visual Metaphors as Complementary Tools for Knowledge Construction and Sharing , 2006, Inf. Vis..

[40]  Daniel Lakens,et al.  An integrative review of the cognitive costs and benefits of note-taking , 2017 .

[41]  Mary Czerwinski,et al.  Toward Characterizing the Productivity Benefits of Very Large Displays , 2003, INTERACT.

[42]  P. Pirolli,et al.  The Sensemaking Process and Leverage Points for Analyst Technology as Identified Through Cognitive Task Analysis , 2015 .

[43]  Jian Zhao,et al.  Supporting Handoff in Asynchronous Collaborative Sensemaking Using Knowledge-Transfer Graphs , 2018, IEEE Transactions on Visualization and Computer Graphics.

[44]  Jiajie Zhang,et al.  The Nature of External Representations in Problem Solving , 1997, Cogn. Sci..

[45]  Ed Huai-hsin Chi,et al.  Entity Workspace: An Evidence File That Aids Memory, Inference, and Reading , 2006, ISI.

[46]  Carol A. Carrier,et al.  The effects of notetaking: A review of studies , 1979 .

[47]  John T. Stasko,et al.  Jigsaw: Supporting Investigative Analysis through Interactive Visualization , 2007, 2007 IEEE Symposium on Visual Analytics Science and Technology.

[48]  Christopher Andrews,et al.  Large high resolution displays for co-located collaborative sensemaking: Display usage and territoriality , 2013, Int. J. Hum. Comput. Stud..

[49]  Stuart K. Card,et al.  The cost structure of sensemaking , 1993, INTERCHI.

[50]  John T. Stasko,et al.  Characterizing the intelligence analysis process: Informing visual analytics design through a longitudinal field study , 2011, 2011 IEEE Conference on Visual Analytics Science and Technology (VAST).

[51]  Chris North,et al.  SAViL: cross-display visual links for sensemaking in display ecologies , 2018, Personal and Ubiquitous Computing.