Space to Think: Sensemaking and Large, High-Resolution Displays

Display technology has developed significantly over the last decade, and it is becoming increasingly feasible to construct large, high-resolution displays. Prior work has shown a number of key performance advantages for these displays that can largely be attributed to the replacement of virtual navigation (e.g., panning and zooming) with physical navigation (e.g., moving, turning, glancing). This research moves beyond the question of performance or efficiency and examines ways in which the large, high-resolution display can support the cognitive demanding task of sensemaking. The core contribution of this work is to show that the physical properties of large, highresolution displays create a fundamentally different environment from conventional displays, one that is inherently spatial, resulting in support for a greater range of embodied resources. To support this, we describe a series of studies that examined the process of sensemaking on one of these displays. These studies illustrate how the display becomes a cognitive partner of the the analyst, encouraging the use of the space for the externalization of the analyst’s thought process or findings. We particularly highlight how the flexibility of the space supports the use of incremental formalism, a process of gradually structuring information as understanding grows. Building on these observations, we have developed a new sensemaking environment called Analyst’s Workspace (AW), which makes use of a large, high-resolution display as a core component of its design. The primary goal of AW is to provide an environment that unifies the activities of foraging and synthesis into a single investigative thread. AW addresses this goal through the use of an integrated spatial environment in which full text documents serve as primary sources of information, investigative tools for pursuing leads, and sensemaking artifacts that can be arranged in the space to encode information about relationships between events and entities. This work also provides a collection of design principles that fell out of the development of AW, and that we hope can guide future development of analytic tools on large, high-resolution displays. This research is partially supported by the National Science Foundation grants CCF-0937133, CNS-04-23611 and IIS-0736055, by the National Geospatial-Intelligence Agency contract HM158205-1-2001, the National Visual Analytics Center (NVAC) and Information Infrastructure Integrity Initiative (I4) at Pacific Northwest National Laboratories, and the Institute for Critical Technology and Applied Science (ICTAS), Virginia Tech.

[1]  Randall H. Trigg,et al.  Notecards in a nutshell , 1986, CHI '87.

[2]  Stephen M. Hess,et al.  The Utility of Display Space in Keeping Track of Rapidly Changing Information , 1999, Hum. Factors.

[3]  Shingo Takada,et al.  Two-dimensional spatial positioning as a means for reflection in design , 2000, DIS '00.

[4]  Frank M. Shipman,et al.  Finding and using implicit structure in human-organized spatial layouts of information , 1995, CHI '95.

[5]  Benjamin B. Bederson,et al.  Does animation help users build mental maps of spatial information? , 1999, Proceedings 1999 IEEE Symposium on Information Visualization (InfoVis'99).

[6]  James D. Hollan,et al.  Spatial Tools for Managing Personal Information Collections , 2005, Proceedings of the 38th Annual Hawaii International Conference on System Sciences.

[7]  Frank M. Shipman,et al.  Spatial hypertext and the practice of information triage , 1997, HYPERTEXT '97.

[8]  Hiroshi Ishii,et al.  A comparison of spatial organization strategies in graphical and tangible user interfaces , 2000, DARE '00.

[9]  Jiajie Zhang,et al.  Representations in Distributed Cognitive Tasks , 1994, Cogn. Sci..

[10]  Frank M. Shipman,et al.  Spatial hypertext: an alternative to navigational and semantic links , 1999, CSUR.

[11]  Benjamin B. Bederson,et al.  A review of overview+detail, zooming, and focus+context interfaces , 2009, CSUR.

[12]  G. W. Furnas,et al.  Generalized fisheye views , 1986, CHI '86.

[13]  John T. Stasko,et al.  Evaluating visual analytics systems for investigative analysis: Deriving design principles from a case study , 2009, 2009 IEEE Symposium on Visual Analytics Science and Technology.

[14]  A. Clark An embodied cognitive science? , 1999, Trends in Cognitive Sciences.

[15]  Werner Kuhn Handling Data Spatially: Spatializating User Interfaces , 1996 .

[16]  John T. Stasko,et al.  Distributed Cognition as a Theoretical Framework for Information Visualization , 2008, IEEE Transactions on Visualization and Computer Graphics.

[17]  Ted Boren,et al.  Thinking aloud: reconciling theory and practice , 2000 .

[18]  Ed Huai-hsin Chi,et al.  Entity Workspace: An Evidence File That Aids Memory, Inference, and Reading , 2006, ISI.

[19]  Jean Scholtz,et al.  Evaluating Visual Analytics at the 2007 VAST Symposium Contest , 2008, IEEE Computer Graphics and Applications.

[20]  Susan T. Dumais,et al.  The spatial metaphor for user interfaces: experimental tests of reference by location versus name , 1986, TOIS.

[21]  qcMIZCV QoKNQ,et al.  Increased Display Size and Resolution Improve Task Performance in Information-Rich Virtual Environments , 2006 .

[22]  Adam M. Fass,et al.  MessyDesk and MessyBoard: two designs inspired by the goal of improving human memory , 2002, DIS '02.

[23]  Mary Czerwinski,et al.  Toward Characterizing the Productivity Benefits of Very Large Displays , 2003, INTERACT.

[24]  Frank M. Shipman,et al.  Formality Considered Harmful: Experiences, Emerging Themes, and Directions on the Use of Formal Representations in Interactive Systems , 1999, Computer Supported Cooperative Work (CSCW).

[25]  Stuart K. Card,et al.  Entity-based collaboration tools for intelligence analysis , 2008, 2008 IEEE Symposium on Visual Analytics Science and Technology.

[26]  Frank Biocca,et al.  How do users organize virtual tools around their body in immersive virtual and augmented environments ? : An exploratory study of egocentric spatial mapping of virtual tools in the mobile infosphere , 2003 .

[27]  S. H. Sato,et al.  Interaction design for large displays , 1997, INTR.

[28]  Patrick Baudisch,et al.  Keeping things in context: a comparative evaluation of focus plus context screens, overviews, and zooming , 2002, CHI.

[29]  Mikkel Rønne Jakobsen,et al.  Sizing up visualizations: effects of display size in focus+context, overview+detail, and zooming interfaces , 2011, CHI.

[30]  Jean Scholtz,et al.  Advancing User-Centered Evaluation of Visual Analytic Environments through Contests , 2009, Inf. Vis..

[31]  Stuart K. Card,et al.  The cost structure of sensemaking , 1993, INTERCHI.

[32]  Mary Czerwinski,et al.  Data mountain: using spatial memory for document management , 1998, UIST '98.

[33]  Chris North,et al.  High-resolution gaming: Interfaces, notifications, and the user experience , 2007, Interact. Comput..

[34]  Richard May,et al.  Supporting Knowledge Transfer through Decomposable Reasoning Artifacts , 2007, 2007 40th Annual Hawaii International Conference on System Sciences (HICSS'07).

[35]  P. Dourish Embodied Interaction: Exploring the Foundations of a New Approach to HCI , 1999 .

[36]  Anthony C. Robinson,et al.  Collaborative synthesis of visual analytic results , 2008, 2008 IEEE Symposium on Visual Analytics Science and Technology.

[37]  Desney S. Tan,et al.  Physically large displays improve performance on spatial tasks , 2006, TCHI.

[38]  Chris North,et al.  Move to improve: promoting physical navigation to increase user performance with large displays , 2007, CHI.

[39]  Eric D. Ragan,et al.  The Effects of Higher Levels of Immersion on Procedure Memorization Performance and Implications for Educational Virtual Environments , 2010, PRESENCE: Teleoperators and Virtual Environments.

[40]  Rudy Darken,et al.  Wayfinding strategies and behaviors in large virtual worlds , 1996, CHI.

[41]  J. O'Regan,et al.  Solving the "real" mysteries of visual perception: the world as an outside memory. , 1992, Canadian journal of psychology.

[42]  Brian D. Ehret,et al.  Learning where to look: location learning in graphical user interfaces , 2002, CHI.

[43]  Alexander W. Skaburskis,et al.  The Sandbox for analysis: concepts and methods , 2006, CHI.

[44]  Rob Johnson Developing a Taxonomy of Intelligence Analysis Variables , 2003 .

[45]  Catherine C. Marshall,et al.  Aquanet: a hypertext tool to hold your knowledge in place , 1991, HYPERTEXT '91.

[46]  Richards J. Heuer,et al.  Psychology of Intelligence Analysis , 1999 .

[47]  Frank G. Halasz,et al.  Reflections on NoteCards: seven issues for the next generation of hypermedia systems , 1987, CACM.

[48]  Mary Czerwinski,et al.  Display space usage and window management operation comparisons between single monitor and multiple monitor users , 2004, AVI.

[49]  Jonathan Grudin,et al.  Partitioning digital worlds: focal and peripheral awareness in multiple monitor use , 2001, CHI.

[50]  Robert Harper,et al.  Stories in GeoTime , 2007 .

[51]  Frank M. Shipman,et al.  Semantics happen: knowledge building in spatial hypertext , 2002, HYPERTEXT '02.

[52]  Kristin A. Cook,et al.  Illuminating the Path: The Research and Development Agenda for Visual Analytics , 2005 .

[53]  Colin Ware,et al.  Zooming versus multiple window interfaces: Cognitive costs of visual comparisons , 2006, TCHI.

[54]  Benjamin B. Bederson,et al.  Space-scale diagrams: understanding multiscale interfaces , 1995, CHI '95.

[55]  P. Pirolli,et al.  The Sensemaking Process and Leverage Points for Analyst Technology as Identified Through Cognitive Task Analysis , 2007 .

[56]  Dieter Schmalstieg,et al.  Visual links across applications , 2010, Graphics Interface.

[57]  Malcolm Slaney,et al.  Being Literate with Large Document Collections: Observational Studies and Cost Structure Tradeoffs , 2006, Proceedings of the 39th Annual Hawaii International Conference on System Sciences (HICSS'06).

[58]  Andy Cockburn,et al.  Evaluating the effectiveness of spatial memory in 2D and 3D physical and virtual environments , 2002, CHI.

[59]  John T. Stasko,et al.  Jigsaw: Supporting Investigative Analysis through Interactive Visualization , 2007, 2007 IEEE Symposium on Visual Analytics Science and Technology.

[60]  Chris North,et al.  The effects of peripheral vision and physical navigation on large scale visualization , 2008, Graphics Interface.

[61]  Desney S. Tan Exploiting the cognitive and social benefits of physically large displays , 2004 .

[62]  Desney S. Tan,et al.  Women take a wider view , 2002, CHI.

[63]  Xiaojun Bi,et al.  Comparing usage of a large high-resolution display to single or dual desktop displays for daily work , 2009, CHI.

[64]  Andy Cockburn,et al.  Revisiting 2D vs 3D Implications on Spatial Memory , 2004, AUIC.

[65]  Paul P. Maglio,et al.  On Distinguishing Epistemic from Pragmatic Action , 1994, Cogn. Sci..

[66]  Austin Henderson,et al.  Rooms: the use of multiple virtual workspaces to reduce space contention in a window-based graphical user interface , 1986, TOGS.

[67]  Dominique L. Scapin,et al.  What do people recall about their documents?: implications for desktop search tools , 2007, IUI '07.

[68]  Anthony C. Robinson,et al.  Design for synthesis in geovisualization , 2008 .

[69]  G. Lakoff,et al.  Metaphors We Live by , 1981 .

[70]  William Wright,et al.  Information Triage with TRIST , 2005 .

[71]  Chris North,et al.  Shaping the Display of the Future: The Effects of Display Size and Curvature on User Performance and Insights , 2009, Hum. Comput. Interact..

[72]  Daniel C. Richardson,et al.  Representation, space and Hollywood Squares: looking at things that aren't there anymore , 2000, Cognition.

[73]  Jiajie Zhang,et al.  The Nature of External Representations in Problem Solving , 1997, Cogn. Sci..

[74]  David Kirsh,et al.  The Intelligent Use of Space , 1995, Artif. Intell..

[75]  Catherine C. Marshall,et al.  Two years before the mist: experiences with Aquanet , 1993, ECHT '92.

[76]  Chris North,et al.  Beyond visual acuity: the perceptual scalability of information visualizations for large displays , 2007, CHI.

[77]  James D. Hollan,et al.  Pad++: advances in multiscale interfaces , 1994, CHI Conference Companion.

[78]  Yasuhiro Yamamoto,et al.  What is the space for?: the role of space in authoring hypertext representations , 2005, HYPERTEXT '05.

[79]  Desney S. Tan,et al.  The large-display user experience , 2005, IEEE Computer Graphics and Applications.

[80]  Mirko Farina Supersizing the Mind: Embodiment, Action and Cognitive Extension. , 2010 .

[81]  Dugald Ralph Hutchings Making Multiple Monitors More Manageable , 2006 .

[82]  Thomas W. Malone,et al.  How do people organize their desks?: Implications for the design of office information systems , 1983, TOIS.

[83]  Wai-Tat Fu,et al.  Soft constraints in interactive behavior: the case of ignoring perfect knowledge in-the-world for imperfect knowledge in-the-head , 2004, Cogn. Sci..

[84]  Mark Bernstein Collage, composites, construction , 2003, HYPERTEXT '03.

[85]  Frank M. Shipman,et al.  Emergent Structure in Analytic Workspaces: Design and Use of the Visual Knowledge Builder , 2001, INTERACT.

[86]  James J. Thomas,et al.  Visualizing the non-visual: spatial analysis and interaction with information from text documents , 1995, Proceedings of Visualization 1995 Conference.

[87]  Frank M. Shipman,et al.  Parsing and interpreting ambiguous structures in spatial hypermedia , 2005, HYPERTEXT '05.

[88]  William Wright,et al.  GeoTime Information Visualization , 2004, IEEE Symposium on Information Visualization.

[89]  James D. Hollan,et al.  Distributed cognition: toward a new foundation for human-computer interaction research , 2000, TCHI.

[90]  John T. Stasko,et al.  Lightweight task/application performance using single versus multiple monitors: a comparative study , 2008, Graphics Interface.