Design of Visualizations for Human-Information Interaction: A Pattern-Based Framework

Abstract Interest in visualization design has increased in recent years. While there is a large body of existing work from which visualization designers can draw, much of the past research has focused on developing new tools and techniques that are aimed at specific contexts. Less focus has been placed on developing holistic frameworks, models, and theories that can guide visualization design at a general level—a level that transcends domains, data types, users, and other contextual factors. In addition, little emphasis has been placed on the thinking processes of designers, including the concepts that designers use, while they are engaged in a visualization design activity. In this book we present a general, holistic framework that is intended to support visualization design for human-information interaction. The framework is composed of a number of conceptual elements that can aid in design thinking. The core of the framework is a pattern language—consisting of a set of 14 basic, abstract patterns—and a...

[1]  J. Gero.  Computational Models of Innovative and Creative Design Processes , 2000 .

[2]  Ian Dennis,et al.  Cognitive processes in engineering design: a longitudinal study , 1994 .

[3]  Paul R. Calder,et al.  Is This a Pattern? , 2002, IEEE Softw..

[4]  Ben Shneiderman,et al.  The future of interactive systems and the emergence of direct manipulation , 1982 .

[5]  Pat Hanrahan,et al.  Polaris: a system for query, analysis and visualization of multi-dimensional relational databases , 2000, IEEE Symposium on Information Visualization 2000. INFOVIS 2000. Proceedings.

[6]  M. S. T. Carpendale,et al.  Considering Visual Variables as a Basis for Information Visualisation , 2003 .

[7]  Steven K. Feiner,et al.  Visual task characterization for automated visual discourse synthesis , 1998, CHI.

[8]  Paul P. Maglio,et al.  The conceptual structure of information space , 2003 .

[9]  Neff Walker,et al.  A classification of visual representations , 1994, CACM.

[10]  Danah Boyd,et al.  Vizster: visualizing online social networks , 2005, IEEE Symposium on Information Visualization, 2005. INFOVIS 2005..

[11]  Shaaron Ainsworth,et al.  The functions of multiple representations , 1999, Comput. Educ..

[12]  Jeffrey Heer,et al.  A tour through the visualization zoo , 2010, Commun. ACM.

[13]  Kamran Sedig,et al.  Adjustable properties of visual representations: Improving the quality of human‐information interaction , 2014, J. Assoc. Inf. Sci. Technol..

[14]  Nigel Cross,et al.  Design method and scientific method , 1981 .

[15]  John T. Stasko,et al.  The Science of Interaction , 2009, Inf. Vis..

[16]  Abraham Arcavi,et al.  The role of visual representations in the learning of mathematics , 2003 .

[17]  Robert McGill,et al.  An Experiment in Graphical Perception , 1986, Int. J. Man Mach. Stud..

[18]  Jean-Daniel Fekete,et al.  NodeTrix: a Hybrid Visualization of Social Networks , 2007, IEEE Transactions on Visualization and Computer Graphics.

[19]  Mary Czerwinski,et al.  Design Study of LineSets, a Novel Set Visualization Technique , 2011, IEEE Transactions on Visualization and Computer Graphics.

[20]  W. Cleveland,et al.  Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods , 1984 .

[21]  L. Boroditsky.  Does Language Shape Thought?: Mandarin and English Speakers' Conceptions of Time , 2001, Cognitive Psychology.

[22]  John K. Gilbert,et al.  Science teachers' use of visual representations , 2014 .

[23]  Kamran Sedig,et al.  Role of interface manipulation style and scaffolding on cognition and concept learning in learnware , 2001, TCHI.

[24]  Heidrun Schumann,et al.  A Design Space of Visualization Tasks , 2013, IEEE Transactions on Visualization and Computer Graphics.

[25]  P. Shannon,et al.  Cytoscape: a software environment for integrated models of biomolecular interaction networks. , 2003, Genome research.

[26]  Kamran Sedig,et al.  Interaction design and cognitive gameplay: role of activation time , 2014, CHI PLAY.

[27]  Philip Brey,et al.  The Epistemology and Ontology of Human-Computer Interaction , 2005, Minds and Machines.

[28]  Andrew M. Dearden,et al.  Pattern Languages in HCI: A Critical Review , 2006, Hum. Comput. Interact..

[29]  Daniel Kahneman,et al.  Availability: A heuristic for judging frequency and probability , 1973 .

[30]  Peter J. Mucha,et al.  Portrait of Political Party Polarization1 , 2013, Network Science.

[31]  Ute Schmid,et al.  The challenge of complexity for cognitive systems , 2011, Cognitive Systems Research.

[32]  David Kirsh,et al.  Embodied cognition and the magical future of interaction design , 2013, TCHI.

[33]  Kamran Sedig,et al.  Adjusting degree of visual complexity: an interactive approach for exploring four-dimensional polytopes , 2004, The Visual Computer.

[34]  Steven F. Roth,et al.  Data characterization for intelligent graphics presentation , 1990, CHI '90.

[35]  John S. Gero,et al.  The structure of concurrent cognitive actions: a case study on novice and expert designers , 2002 .

[36]  Ed H. Chi,et al.  A taxonomy of visualization techniques using the data state reference model , 2000, IEEE Symposium on Information Visualization 2000. INFOVIS 2000. Proceedings.

[37]  David Kirsh,et al.  Thinking with external representations , 2010, AI & SOCIETY.

[38]  Lucy T. Nowell,et al.  ThemeRiver: visualizing theme changes over time , 2000, IEEE Symposium on Information Visualization 2000. INFOVIS 2000. Proceedings.

[39]  Hai-Ning Liang,et al.  Application of Information Visualization Techniques to the Design of a Mathematical Mindtool: A Usability Study , 2003, Inf. Vis..

[40]  Hai-Ning Liang,et al.  Interactivity of Visual Mathematical Representations: Factors Affecting Learning and Cognitive Processes , 2006 .

[41]  Hernan Casakin,et al.  Assessing the Use of Metaphors in the Design Process , 2006 .

[42]  M. Hegarty,et al.  Types of visual–spatial representations and mathematical problem solving. , 1999 .

[43]  Niklas Elmqvist,et al.  Patterns for visualization evaluation , 2015, Inf. Vis..

[44]  David H. Jonassen,et al.  Computers as cognitive tools: Learningwith technology, notfrom technology , 1995, J. Comput. High. Educ..

[45]  John T. Stasko,et al.  Distributed Cognition as a Theoretical Framework for Information Visualization , 2008, IEEE Transactions on Visualization and Computer Graphics.

[46]  Vimla L. Patel,et al.  Distributed cognition, representation, and affordance , 2006 .

[47]  Robert E. Mercer,et al.  Interactive metamorphic visuals: exploring polyhedral relationships , 2001, Proceedings Fifth International Conference on Information Visualisation.

[48]  Hai-Ning Liang,et al.  Learner-Information Interaction: A Macro-Level Framework Characterizing Visual Cognitive Tools. , 2008 .

[49]  N. Hari Narayanan,et al.  Multimedia design for communication of dynamic information , 2002, Int. J. Hum. Comput. Stud..

[50]  James R. Eagan,et al.  Low-level components of analytic activity in information visualization , 2005, IEEE Symposium on Information Visualization, 2005. INFOVIS 2005..

[51]  Gunnvald B. Svendsen,et al.  The Influence of Interface Style on Problem Solving , 1991, Int. J. Man Mach. Stud..

[52]  Hai-Ning Liang,et al.  Designing interfaces that support formation of cognitive maps of transitional processes: an empirical study , 2005, Interact. Comput..

[53]  Douglas C. Schmidt,et al.  Using design patterns to develop reusable object-oriented communication software , 1995, CACM.

[54]  Jock D. Mackinlay,et al.  Automating the design of graphical presentations of relational information , 1986, TOGS.

[55]  Jeffrey Heer,et al.  Crowdsourcing graphical perception: using mechanical turk to assess visualization design , 2010, CHI.

[56]  J.C. Roberts,et al.  State of the Art: Coordinated & Multiple Views in Exploratory Visualization , 2007, Fifth International Conference on Coordinated and Multiple Views in Exploratory Visualization (CMV 2007).

[57]  John M. Carroll,et al.  Making use: scenarios and scenario-based design , 2000, DIS '00.

[58]  Yuri Engelhardt.  Objects and Spaces: The Visual Language of Graphics , 2006, Diagrams.

[59]  Ben Shneiderman,et al.  Interactive Dynamics for Visual Analysis , 2012 .

[60]  Richard Coyne,et al.  Models, Metaphors and the Hermeneutics of Designing , 1992 .

[61]  Jie Lu,et al.  HARVEST: an intelligent visual analytic tool for the masses , 2010 .

[62]  Brian D. Fisher,et al.  Visual Analytics as a Translational Cognitive Science , 2011, Top. Cogn. Sci..

[63]  Markus Knauff,et al.  Complex cognition: the science of human reasoning, problem-solving, and decision-making , 2010, Cognitive Processing.

[64]  David Kirsh,et al.  The importance of chance and interactivity in creativity , 2014 .

[65]  Wilmot Li,et al.  Design principles for visual communication , 2011, Commun. ACM.

[66]  Donald A. Norman,et al.  Human-centered design considered harmful , 2005, INTR.

[67]  Jiajie Zhang,et al.  Representations in Distributed Cognitive Tasks , 1994, Cogn. Sci..

[68]  Alan J. Dix,et al.  Starting simple: adding value to static visualisation through simple interaction , 1998, AVI '98.

[69]  Brian D. Fisher,et al.  The Personal Equation of Complex Individual Cognition during Visual Interface Interaction , 2009, HCIV.

[70]  James D. Hollan,et al.  Direct Manipulation Interfaces , 1985, Hum. Comput. Interact..

[71]  Remo Aslak Burkhard,et al.  Towards a Framework and a Model for Knowledge Visualization: Synergies Between Information and Knowledge Visualization , 2005, Knowledge and Information Visualization.

[72]  Stephen J. Payne,et al.  Reflection and goal management in exploratory learning , 1995, Int. J. Hum. Comput. Stud..

[73]  Arvind Satyanarayan,et al.  Lyra: An Interactive Visualization Design Environment , 2014, Comput. Graph. Forum.

[74]  M. Sheelagh T. Carpendale,et al.  Visualizing explicit and implicit relations of complex information spaces , 2012, Inf. Vis..

[75]  Nikos A. Salingaros,et al.  Architecture, Patterns, and Mathematics , 1999 .

[76]  David E. Kieras,et al.  Towards demystification of direct manipulation: cognitive modeling charts the gulf of execution , 2001, CHI.

[77]  Jon Oberlander,et al.  A cognitive theory of graphical and linguistic reasoning: logic and implementation. Cognitive Science , 1995 .

[78]  Rachel K. E. Bellamy,et al.  Using the “Physics” of notations to analyze a visual representation of business decision modeling , 2012, 2012 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC).

[79]  Chaim Zins,et al.  Conceptual approaches for defining data, information, and knowledge , 2007, J. Assoc. Inf. Sci. Technol..

[80]  Nikos A. Salingaros,et al.  The structure of pattern languages , 2000 .

[81]  Marti A. Hearst.  TileBars: visualization of term distribution information in full text information access , 1995, CHI '95.

[82]  Donald A. Norman,et al.  Affordance, conventions, and design , 1999, INTR.

[83]  Mark Sumner,et al.  Characterizing Interaction with Visual Mathematical Representations , 2006, Int. J. Comput. Math. Learn..

[84]  Jeffrey Heer,et al.  Software Design Patterns for Information Visualization , 2006, IEEE Transactions on Visualization and Computer Graphics.

[85]  Thomas G. Dietterich,et al.  A model of the mechanical design process based on empirical data , 1988, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[86]  Claudia Iacob,et al.  A design pattern mining method for interaction design , 2011, EICS '11.

[87]  M. Teresa Cabré Castellví,et al.  Theories of terminology. Their description, prescription and explanation , 2003 .

[88]  Min Chen,et al.  Data, Information, and Knowledge in Visualization , 2009, IEEE Computer Graphics and Applications.

[89]  L. Boroditsky,et al.  Metaphors We Think With: The Role of Metaphor in Reasoning , 2011, PloS one.

[90]  D. Perkins,et al.  Partners in Cognition: Extending Human Intelligence with Intelligent Technologies , 1991 .

[91]  Allison Woodruff,et al.  Guidelines for using multiple views in information visualization , 2000, AVI '00.

[92]  P. Cheng.  Electrifying diagrams for learning: principles for effective representational systems , 2002 .

[93]  R. Kozma.  The material features of multiple representations and their cognitive and social affordances for science understanding , 2003 .

[94]  Tamara Munzner,et al.  Design Study Methodology: Reflections from the Trenches and the Stacks , 2012, IEEE Transactions on Visualization and Computer Graphics.

[95]  James D. Hollan,et al.  Distributed cognition: toward a new foundation for human-computer interaction research , 2000, TCHI.

[96]  Jaime Snyder,et al.  Visual representation of information as communicative practice , 2014, J. Assoc. Inf. Sci. Technol..

[97]  Kamran Sedig,et al.  Transactions on Human-computer Interaction Thci Design for Complex Cognitive Activities with Visual Representations: a Pattern-based Approach Sedig and Parsons Interaction Design for Complex Cognitive Activities with Visualizations , 2022 .

[98]  Shaaron Ainsworth,et al.  The Educational Value of Multiple-representations when Learning Complex Scientific Concepts , 2008 .

[99]  Niklas Elmqvist,et al.  Fluid interaction for information visualization , 2011, Inf. Vis..

[100]  Jan O. Borchers.  A pattern approach to interaction design , 2001, DIS '00.

[101]  Tamara Munzner,et al.  A Nested Model for Visualization Design and Validation , 2009, IEEE Transactions on Visualization and Computer Graphics.

[102]  Silvia Miksch,et al.  Task Cube: A three-dimensional conceptual space of user tasks in visualization design and evaluation , 2016, Inf. Vis..

[103]  Deborah Hix,et al.  Graphical encoding for information visualization: using icon color, shape, and size to convey nominal and quantitative data , 1997 .

[104]  Robert E. Mercer,et al.  18 Visualising, interacting and experimenting with lattices using a diagrammatic representation , 2005 .

[105]  Gerd Fricke,et al.  Successful individual approaches in engineering design , 1996 .

[106]  Lucy T. Nowell.  Graphical encoding in information visualization , 1997, CHI Extended Abstracts.

[107]  Hai-Ning Liang,et al.  Role of Interaction in Enhancing the Epistemic Utility of 3D Mathematical Visualizations , 2010, Int. J. Comput. Math. Learn..

[108]  Andrew Koenig,et al.  Patterns and Antipatterns , 1998, J. Object Oriented Program..

[109]  Paul P. Maglio,et al.  On Distinguishing Epistemic from Pragmatic Action , 1994, Cogn. Sci..

[110]  Jeffrey Heer,et al.  Protovis: A Graphical Toolkit for Visualization , 2009, IEEE Transactions on Visualization and Computer Graphics.

[111]  Ramana Rao,et al.  The table lens: merging graphical and symbolic representations in an interactive focus + context visualization for tabular information , 1994, CHI '94.

[112]  Rebecca E. Grinter,et al.  Putting it all together: towards a pattern language for interaction design: A CHI 97 workshop , 1998, SGCH.

[113]  Heidrun Schumann,et al.  A Flexible Approach for Visual Data Mining , 2002, IEEE Trans. Vis. Comput. Graph..

[114]  John R. Anderson,et al.  Locus of feedback control in computer-based tutoring: impact on learning rate, achievement and attitudes , 2001, CHI.

[115]  Andrew Dillon.  Spatial-semantics: how users derive shape from information space , 2000 .

[116]  Erik Stolterman,et al.  The Nature of Design Practice and Implications for Interaction Design Research , 2008 .

[117]  C M Carswell,et al.  Choosing Specifiers: An Evaluation of the Basic Tasks Model of Graphical Perception , 1992, Human factors.

[118]  Antonella De Angeli,et al.  Do patterns help novice evaluators? A comparative study , 2011, Int. J. Hum. Comput. Stud..

[119]  Silvia Miksch,et al.  Radial Sets: Interactive Visual Analysis of Large Overlapping Sets , 2013, IEEE Transactions on Visualization and Computer Graphics.

[120]  Willemien Visser,et al.  More or Less Following a Plan During Design: Opportunistic Deviations in Specification , 1990, Int. J. Man Mach. Stud..

[121]  DAVID KIRSH,et al.  Interactivity and multimedia interfaces , 1997 .

[122]  Jeffrey Heer,et al.  The Effects of Interactive Latency on Exploratory Visual Analysis , 2014, IEEE Transactions on Visualization and Computer Graphics.

[123]  S. Ainsworth.  DeFT: A Conceptual Framework for Considering Learning with Multiple Representations. , 2006 .

[124]  Michael Burch,et al.  TimeRadarTrees: Visualizing Dynamic Compound Digraphs , 2008, Comput. Graph. Forum.

[125]  Mary Hegarty,et al.  The Cognitive Science of Visual-Spatial Displays: Implications for Design , 2011, Top. Cogn. Sci..

[126]  Robert E. Roth,et al.  An Empirically-Derived Taxonomy of Interaction Primitives for Interactive Cartography and Geovisualization , 2013, IEEE Transactions on Visualization and Computer Graphics.

[127]  Martin Reisslein,et al.  Teaching with Concrete and Abstract Visual Representations: Effects on Students' Problem Solving, Problem Representations, and Learning Perceptions. , 2011 .

[128]  Josef Pfeuffer,et al.  Combined passive and active shimming for in vivo MR spectroscopy at high magnetic fields. , 2006, Journal of magnetic resonance.