Characterizing the Quality of Insight by Interactions: A Case Study

Understanding the quality of insight has become increasingly important with the trend of allowing users to post comments during visual exploration, yet approaches for qualifying insight are rare. This paper presents a case study to investigate the possibility of characterizing the quality of insight via the interactions performed. To do this, we devised the interaction of a visualization tool-MediSyn-for insight generation. MediSyn supports five types of interactions: selecting, connecting, elaborating, exploring, and sharing. We evaluated MediSyn with 14 participants by allowing them to freely explore the data and generate insights. We then extracted seven interaction patterns from their interaction logs and correlated the patterns to four aspects of insight quality. The results show the possibility of qualifying insights via interactions. Among other findings, exploration actions can lead to unexpected insights; the drill-down pattern tends to increase the domain values of insights. A qualitative analysis shows that using domain knowledge to guide exploration can positively affect the domain value of derived insights. We discuss the study's implications, lessons learned, and future research opportunities.

[1]  Bongshin Lee,et al.  GraphTrail: analyzing large multivariate, heterogeneous networks while supporting exploration history , 2012, CHI.

[2]  Tuukka Ruotsalo,et al.  From Hyperlinks to Hypercues: Entity-Based Affordances for Fluid Information Exploration , 2018, Conference on Designing Interactive Systems.

[3]  Chris North,et al.  Semantic Interaction: Coupling Cognition and Computation through Usable Interactive Analytics , 2015, IEEE Computer Graphics and Applications.

[4]  Cheryl Z. Qian,et al.  Capturing and supporting the analysis process , 2009, 2009 IEEE Symposium on Visual Analytics Science and Technology.

[5]  John T. Stasko,et al.  Jigsaw: Supporting Investigative Analysis through Interactive Visualization , 2007, 2007 IEEE Symposium on Visual Analytics Science and Technology.

[6]  Bob Fields,et al.  SensePath: Understanding the Sensemaking Process Through Analytic Provenance , 2016, IEEE Transactions on Visualization and Computer Graphics.

[7]  Eser Kandogan,et al.  Towards a Unified Representation of Insight in Human-in-the-Loop Analytics: A User Study , 2018, HILDA@SIGMOD.

[8]  William Ribarsky,et al.  Recovering Reasoning Processes from User Interactions , 2009, IEEE Computer Graphics and Applications.

[9]  Jeffrey Heer,et al.  Graphical Histories for Visualization: Supporting Analysis, Communication, and Evaluation , 2008, IEEE Transactions on Visualization and Computer Graphics.

[10]  Jarke J. van Wijk,et al.  Supporting the analytical reasoning process in information visualization , 2008, CHI.

[11]  Jian Zhao,et al.  Annotation Graphs: A Graph-Based Visualization for Meta-Analysis of Data Based on User-Authored Annotations , 2017, IEEE Transactions on Visualization and Computer Graphics.

[12]  Ilo Alexandre,et al.  Promoting Insight: A Case Study of How to Incorporate Interaction in Existing Data Visualizations , 2016, 2016 20th International Conference Information Visualisation (IV).

[13]  Martin Wattenberg,et al.  Voyagers and voyeurs: Supporting asynchronous collaborative visualization , 2009, CACM.

[14]  David H. Laidlaw,et al.  An insight- and task-based methodology for evaluating spatiotemporal visual analytics , 2014, 2014 IEEE Conference on Visual Analytics Science and Technology (VAST).

[15]  Dennis P. Groth,et al.  Provenance and Annotation for Visual Exploration Systems , 2006, IEEE Transactions on Visualization and Computer Graphics.

[16]  Lyndsey Franklin,et al.  The Scalable Reasoning System: Lightweight visualization for distributed analytics , 2008, 2008 IEEE Symposium on Visual Analytics Science and Technology.

[17]  Michael E. Papka,et al.  Modeling and evaluating user behavior in exploratory visual analysis , 2016, Inf. Vis..

[18]  Chris North,et al.  An Insight-Based Longitudinal Study of Visual Analytics , 2006, IEEE Transactions on Visualization and Computer Graphics.

[19]  Thomas Ertl,et al.  VA2: A Visual Analytics Approach for Evaluating Visual Analytics Applications , 2016, IEEE Transactions on Visualization and Computer Graphics.

[20]  Cláudio T. Silva,et al.  VisTrails: visualization meets data management , 2006, SIGMOD Conference.

[21]  Marti A. Hearst,et al.  Futzing and Moseying: Interviews with Professional Data Analysts on Exploration Practices , 2019, IEEE Transactions on Visualization and Computer Graphics.

[22]  Silvia Miksch,et al.  Analysing Interactivity in Information Visualisation , 2012, KI - Künstliche Intelligenz.

[23]  Karthic Madanagopal,et al.  Analytic Provenance in Practice: The Role of Provenance in Real-World Visualization and Data Analysis Environments , 2019, IEEE Computer Graphics and Applications.

[24]  Kate Herd,et al.  SenseMap: Supporting browser-based online sensemaking through analytic provenance , 2016, 2016 IEEE Conference on Visual Analytics Science and Technology (VAST).

[25]  William Ribarsky,et al.  Capturing Reasoning Process through User Interaction , 2010, EuroVAST@EuroVis.

[26]  Min Chen,et al.  Data, Information, and Knowledge in Visualization , 2009, IEEE Computer Graphics and Applications.

[27]  Jesus J. Caban,et al.  A Grammar-based Approach for Modeling User Interactions and Generating Suggestions During the Data Exploration Process , 2017, IEEE Transactions on Visualization and Computer Graphics.

[28]  Lane Harrison,et al.  Patterns and Pace: Quantifying Diverse Exploration Behavior with Visualizations on the Web , 2019, IEEE Transactions on Visualization and Computer Graphics.

[29]  Charles Perin,et al.  Exploration Strategies for Discovery of Interactivity in Visualizations , 2019, IEEE Transactions on Visualization and Computer Graphics.

[30]  John T. Stasko,et al.  Mental Models, Visual Reasoning and Interaction in Information Visualization: A Top-down Perspective , 2010, IEEE Transactions on Visualization and Computer Graphics.

[31]  Brian D. Fisher,et al.  Towards the Personal Equation of Interaction: The impact of personality factors on visual analytics interface interaction , 2010, 2010 IEEE Symposium on Visual Analytics Science and Technology.

[32]  Chris North,et al.  An insight-based methodology for evaluating bioinformatics visualizations , 2005, IEEE Transactions on Visualization and Computer Graphics.

[33]  Leilani Battle,et al.  Characterizing Exploratory Visual Analysis: A Literature Review and Evaluation of Analytic Provenance in Tableau , 2019, Comput. Graph. Forum.

[34]  Hafeez Ur Rehman,et al.  Integration and Analysis of Heterogeneous Biological Data , 2014 .

[35]  Bongshin Lee,et al.  Inking Your Insights: Investigating Digital Externalization Behaviors During Data Analysis , 2019, ISS.

[36]  John T. Stasko,et al.  Toward a Deeper Understanding of the Role of Interaction in Information Visualization , 2007, IEEE Transactions on Visualization and Computer Graphics.

[37]  Chris North,et al.  Analytic provenance: process+interaction+insight , 2011, CHI Extended Abstracts.

[38]  Yang Wang,et al.  Patterns and Sequences: Interactive Exploration of Clickstreams to Understand Common Visitor Paths , 2017, IEEE Transactions on Visualization and Computer Graphics.

[39]  Eva Mayr,et al.  Many roads lead to Rome: mapping users' problem solving strategies , 2010, BELIV '10.

[40]  P. Pirolli,et al.  The Sensemaking Process and Leverage Points for Analyst Technology as Identified Through Cognitive Task Analysis , 2015 .

[41]  Jian Zhao,et al.  Supporting Handoff in Asynchronous Collaborative Sensemaking Using Knowledge-Transfer Graphs , 2018, IEEE Transactions on Visualization and Computer Graphics.

[42]  Chris North,et al.  A comparison of benchmark task and insight evaluation methods for information visualization , 2011, Inf. Vis..

[43]  Ben Shneiderman,et al.  Interactive Dynamics for Visual Analysis , 2012 .

[44]  Bongshin Lee,et al.  Characterizing Visualization Insights from Quantified Selfers' Personal Data Presentations , 2015, IEEE Computer Graphics and Applications.

[45]  Alex Endert,et al.  Characterizing Provenance in Visualization and Data Analysis: An Organizational Framework of Provenance Types and Purposes , 2016, IEEE Transactions on Visualization and Computer Graphics.

[46]  Tero Aittokallio,et al.  MediSyn: uncertainty-aware visualization of multiple biomedical datasets to support drug treatment selection , 2017, BMC Bioinformatics.

[47]  Susan T. Dumais,et al.  PivotPaths: Strolling through Faceted Information Spaces , 2012, IEEE Transactions on Visualization and Computer Graphics.

[48]  Shimei Pan,et al.  Analytic Trails: Supporting Provenance, Collaboration, and Reuse for Visual Data Analysis by Business Users , 2011, INTERACT.

[49]  Alex Endert,et al.  Finding Waldo: Learning about Users from their Interactions , 2014, IEEE Transactions on Visualization and Computer Graphics.

[50]  Niklas Elmqvist,et al.  DataSite: Proactive visual data exploration with computation of insight-based recommendations , 2018, Inf. Vis..

[51]  Stuart K. Card,et al.  Entity-based collaboration tools for intelligence analysis , 2008, 2008 IEEE Symposium on Visual Analytics Science and Technology.

[52]  David H. Laidlaw,et al.  A Case Study Using Visualization Interaction Logs and Insight Metrics to Understand How Analysts Arrive at Insights , 2016, IEEE Transactions on Visualization and Computer Graphics.

[53]  T. J. Jankun-Kelly,et al.  Analytic Provenance for Sensemaking: A Research Agenda , 2015, IEEE Computer Graphics and Applications.

[54]  Silvia Miksch,et al.  To Score or Not to Score? Tripling Insights for Participatory Design , 2009, IEEE Computer Graphics and Applications.

[55]  Björn Hartmann,et al.  Identifying Redundancy and Exposing Provenance in Crowdsourced Data Analysis , 2013, IEEE Transactions on Visualization and Computer Graphics.

[56]  Michelle X. Zhou,et al.  Characterizing Users' Visual Analytic Activity for Insight Provenance , 2008, 2008 IEEE Symposium on Visual Analytics Science and Technology.

[57]  John T. Stasko,et al.  Understanding and characterizing insights: how do people gain insights using information visualization? , 2008, BELIV.

[58]  Heather Richter Lipford,et al.  Helping users recall their reasoning process , 2010, 2010 IEEE Symposium on Visual Analytics Science and Technology.

[59]  Zhen Wen,et al.  Behavior-driven visualization recommendation , 2009, IUI.

[60]  Martin Wattenberg,et al.  ManyEyes: a Site for Visualization at Internet Scale , 2007, IEEE Transactions on Visualization and Computer Graphics.

[61]  Michelle X. Zhou,et al.  Understand users’ comprehension and preferences for composing information visualizations , 2014, TCHI.

[62]  Clemens Nylandsted Klokmose,et al.  InsideInsights: Integrating Data‐Driven Reporting in Collaborative Visual Analytics , 2019, Comput. Graph. Forum.

[63]  Chris North,et al.  Toward measuring visualization insight , 2006, IEEE Computer Graphics and Applications.

[64]  William Ribarsky,et al.  Defining and applying knowledge conversion processes to a visual analytics system , 2009, Comput. Graph..

[65]  Thomas Ertl,et al.  Visual analysis and coding of data-rich user behavior , 2016, 2016 IEEE Conference on Visual Analytics Science and Technology (VAST).

[66]  H. Theisel,et al.  ExPlates: Spatializing Interactive Analysis to Scaffold Visual Exploration , 2013, Comput. Graph. Forum.

[67]  David Gotz,et al.  Connecting the dots in visual analysis , 2009, 2009 IEEE Symposium on Visual Analytics Science and Technology.

[68]  Michael Gertz,et al.  A Model and Framework for Visualization Exploration , 2007, IEEE Transactions on Visualization and Computer Graphics.

[69]  Wenwen Dou,et al.  A Survey on Interaction Log Analysis for Evaluating Exploratory Visualizations , 2016, BELIV '16.

[70]  John T. Stasko,et al.  The Science of Interaction , 2009, Inf. Vis..

[71]  William Ribarsky,et al.  Defining Insight for Visual Analytics , 2009, IEEE Computer Graphics and Applications.