Storytelling and Visualization: An Extended Survey

Throughout history, storytelling has been an effective way of conveying information and knowledge. In the field of visualization, storytelling is rapidly gaining momentum and evolving cutting-edge techniques that enhance understanding. Many communities have commented on the importance of storytelling in data visualization. Storytellers tend to be integrating complex visualizations into their narratives in growing numbers. In this paper, we present a survey of storytelling literature in visualization and present an overview of the common and important elements in storytelling visualization. We also describe the challenges in this field as well as a novel classification of the literature on storytelling in visualization. Our classification scheme highlights the open and unsolved problems in this field as well as the more mature storytelling sub-fields. The benefits offer a concise overview and a starting point into this rapidly evolving research trend and provide a deeper understanding of this topic.

[1]  Adrian Kuhn,et al.  CodeTimeline: Storytelling with versioning data , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[2]  Nicholas Diakopoulos,et al.  Contextifier: automatic generation of annotated stock visualizations , 2013, CHI.

[3]  Kwan-Liu Ma,et al.  Dynamic video narratives , 2010, SIGGRAPH 2010.

[4]  Kwan-Liu Ma,et al.  AniViz: A Template-Based Animation Tool for Volume Visualization , 2010, IEEE Computer Graphics and Applications.

[5]  B. L. William Wong,et al.  SchemaLine: Timeline Visualization for Sensemaking , 2014, 2014 18th International Conference on Information Visualisation.

[6]  Robert S. Laramee,et al.  Storytelling and Visualization: A Survey , 2018, VISIGRAPP.

[7]  Linda B. Gambrell,et al.  Mental imagery, text illustrations, and children's story comprehension and recall. , 1993 .

[8]  Danah Boyd,et al.  Digital artifacts for remembering and storytelling: posthistory and social network fragments , 2004, 37th Annual Hawaii International Conference on System Sciences, 2004. Proceedings of the.

[9]  Kwan-Liu Ma,et al.  Scientific Storytelling Using Visualization , 2012, IEEE Computer Graphics and Applications.

[10]  Bongshin Lee,et al.  Authoring Data-Driven Videos with DataClips , 2017, IEEE Transactions on Visualization and Computer Graphics.

[11]  Bongshin Lee,et al.  A Deeper Understanding of Sequence in Narrative Visualization , 2013, IEEE Transactions on Visualization and Computer Graphics.

[12]  Eser Kandogan,et al.  Just-in-time annotation of clusters, outliers, and trends in point-based data visualizations , 2012, 2012 IEEE Conference on Visual Analytics Science and Technology (VAST).

[13]  Kyle Wm. Hall,et al.  Telling Stories about Dynamic Networks with Graph Comics , 2016, CHI.

[14]  M. Bal,et al.  Narratology: Introduction to the Theory of Narrative , 1988 .

[15]  Yizhou Yu,et al.  Anomaly detection in GPS data based on visual analytics , 2010, 2010 IEEE Symposium on Visual Analytics Science and Technology.

[16]  Benoit Huet,et al.  Automatic video summarization , 2006 .

[17]  M. Donald Précis of Origins of the modern mind: Three stages in the evolution of culture and cognition , 1993, Behavioral and Brain Sciences.

[18]  Ivan Viola,et al.  Geological storytelling , 2013, Comput. Graph..

[19]  Johannah Rodgers,et al.  Defining and Experiencing Authorship(s) in the Composition Classroom: Findings from a Qualitative Study of Undergraduate Writing Students at the City University of New York , 2011 .

[20]  John T. Stasko,et al.  Effectiveness of Animation in Trend Visualization , 2008, IEEE Transactions on Visualization and Computer Graphics.

[21]  Nigel Holmes Designer's Guide to Creating Charts and Diagrams , 1984 .

[22]  Andrew Begel,et al.  Codebook: discovering and exploiting relationships in software repositories , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[23]  Antonio Torralba,et al.  Understanding the Intrinsic Memorability of Images , 2011, NIPS.

[24]  Nelson L. Max,et al.  A contract based system for large data visualization , 2005, VIS 05. IEEE Visualization, 2005..

[25]  P. Pirolli,et al.  The Sensemaking Process and Leverage Points for Analyst Technology as Identified Through Cognitive Task Analysis , 2007 .

[26]  Robert S. Laramee How to Read a Visualization Research Paper: Extracting the Essentials , 2011, IEEE Computer Graphics and Applications.

[27]  Stephen G. Kobourov,et al.  Map‐based Visualizations Increase Recall Accuracy of Data , 2015, Comput. Graph. Forum.

[28]  Wei-Ta Chu,et al.  Optimized Comics-Based Storytelling for Temporal Image Sequences , 2015, IEEE Transactions on Multimedia.

[29]  Michelle A. Borkin,et al.  What Makes a Visualization Memorable? , 2013, IEEE Transactions on Visualization and Computer Graphics.

[30]  Jack Zipes,et al.  Creative Storytelling: Building Community/Changing Lives , 1995 .

[31]  Cleotilde Gonzalez,et al.  Does animation in user interfaces improve decision making? , 1996, CHI.

[32]  George M. Kasper,et al.  Animation in user interface design for decision-making: a research framework and empirical analysis , 1995 .

[33]  Robert Harper,et al.  Stories in GeoTime , 2007 .

[34]  S. Phithakkitnukoon,et al.  Urban mobility study using taxi traces , 2011, TDMA '11.

[35]  Mengchen Liu,et al.  StoryFlow: Tracking the Evolution of Stories , 2013, IEEE Transactions on Visualization and Computer Graphics.

[36]  Elaine Cohen,et al.  A non-photorealistic lighting model for automatic technical illustration , 1998, SIGGRAPH.

[37]  Benjamin S. Bloom,et al.  Taxonomy of Educational Objectives: The Classification of Educational Goals. , 1957 .

[38]  Charles Hansen,et al.  The Visualization Handbook , 2011 .

[39]  Carl Gutwin,et al.  Useful junk?: the effects of visual embellishment on comprehension and memorability of charts , 2010, CHI.

[40]  Helwig Hauser,et al.  Story Telling for Presentation in Volume Visualization , 2007, EuroVis.

[41]  Jeffrey Heer,et al.  Animated Transitions in Statistical Data Graphics , 2007, IEEE Transactions on Visualization and Computer Graphics.

[42]  Kwan-Liu Ma,et al.  Storytelling via Navigation: A Novel Approach to Animation for Scientific Visualization , 2014, Smart Graphics.

[43]  Jeffrey Heer,et al.  Narrative Visualization: Telling Stories with Data , 2010, IEEE Transactions on Visualization and Computer Graphics.

[44]  M. Sheelagh T. Carpendale,et al.  More Than Telling a Story: Transforming Data into Visually Shared Stories , 2015, IEEE Computer Graphics and Applications.

[45]  Martin Wattenberg,et al.  Voyagers and voyeurs: supporting asynchronous collaborative information visualization , 2007, CHI.

[46]  Shi-Min Hu,et al.  Visual storylines: Semantic visualization of movie sequence , 2012, Comput. Graph..

[47]  Nicholas Diakopoulos,et al.  Visualization Rhetoric: Framing Effects in Narrative Visualization , 2011, IEEE Transactions on Visualization and Computer Graphics.

[48]  Penousal Machado,et al.  Generative Storytelling for Information Visualization , 2011, IEEE Computer Graphics and Applications.

[49]  Arvind Satyanarayan,et al.  Authoring Narrative Visualizations with Ellipsis , 2014, Comput. Graph. Forum.

[50]  Dirk Heylen,et al.  Generating expressive speech for storytelling applications , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[51]  Kwan-Liu Ma,et al.  Software evolution storylines , 2010, SOFTVIS '10.

[52]  Sung-Hee Kim,et al.  Towards a Taxonomy for Evaluating User Engagement in Information Visualization , 2015 .

[53]  David Jensen,et al.  TimeMines: Constructing Timelines with Statistical Models of Word Usage , 2000, KDD 2000.

[54]  Ivan Viola,et al.  Geological storytelling: graphically exploring and communicating geological sketches , 2012, SBIM '12.

[55]  Mikael Jern,et al.  Geovisual Analytics and Storytelling Using HTML5 , 2013, 2013 17th International Conference on Information Visualisation.

[56]  Lukas Mroz,et al.  RTVR-a flexible Java library for interactive volume rendering , 2001, Proceedings Visualization, 2001. VIS '01..

[57]  George G. Robertson,et al.  Narratives: A visualization to track narrative events as they develop , 2008, 2008 IEEE Symposium on Visual Analytics Science and Technology.

[58]  Brent J. Hecht,et al.  NewsViews: an automated pipeline for creating custom geovisualizations for news , 2014, CHI.

[59]  Jean-Daniel Fekete,et al.  Can Initial Narrative VisualizationTechniques and Storytelling help Engage Online-Users with Exploratory Information Visualizations? , 2015 .

[60]  Paulo Nazareno Maia Sampaio,et al.  MOGRE-Storytelling: Interactive Creation of 3D Stories , 2014, 2014 XVI Symposium on Virtual and Augmented Reality.

[61]  Desney S. Tan,et al.  Phosphor: explaining transitions in the user interface using afterglow effects , 2006, UIST.

[62]  Nahum Gershon,et al.  What storytelling can do for information visualization , 2001, Commun. ACM.

[63]  Jesse Schell,et al.  The Art of Game Design: A book of lenses , 2019 .

[64]  Kwan-Liu Ma,et al.  Design Considerations for Optimizing Storyline Visualizations , 2012, IEEE Transactions on Visualization and Computer Graphics.

[65]  Rosemary Luckin,et al.  Designing multimedia for learning: narrative guidance and narrative construction , 1999, CHI '99.

[66]  Zhen Wen,et al.  Behavior-driven visualization recommendation , 2009, IUI.

[67]  Bongshin Lee,et al.  SketchStory: Telling More Engaging Stories with Data through Freeform Sketching , 2013, IEEE Transactions on Visualization and Computer Graphics.

[68]  Ana Figueiras,et al.  How to Tell Stories Using Visualization , 2014, 2014 18th International Conference on Information Visualisation.

[69]  Jock D. Mackinlay,et al.  Storytelling: The Next Step for Visualization , 2013, Computer.

[70]  Mina Akaishi,et al.  Narrative based Topic Visualization for Chronological Data , 2007, 2007 11th International Conference Information Visualization (IV '07).

[71]  A. D. Cropper,et al.  Scientific Storytelling: From up in the clouds to down to earth… A new approach to mentoring , 2015, 2015 IEEE Integrated STEM Education Conference.

[72]  Barbara Tversky,et al.  Animation: can it facilitate? , 2002, Int. J. Hum. Comput. Stud..

[73]  Paula Alavesa,et al.  Combining Storytelling Tradition and Pervasive Gaming , 2013, 2013 5th International Conference on Games and Virtual Worlds for Serious Applications (VS-GAMES).

[74]  Ivan Viola,et al.  Importance-Driven Expressive Visualization , 2005 .

[75]  Michael Wohlfart,et al.  Story Telling Aspects in Medical Applications , 2006 .

[76]  Robert F. Dennehy The Springboard: How Storytelling Ignites Action in Knowledge‐era Organizations , 2001 .

[77]  Kwan-Liu Ma,et al.  code_swarm: A Design Study in Organic Software Visualization , 2009, IEEE Transactions on Visualization and Computer Graphics.

[78]  Han-Wei Shen,et al.  Interactive Storyboard for Overall Time-Varying Data Visualization , 2008, 2008 IEEE Pacific Visualization Symposium.

[79]  Alexander Lex,et al.  From Visual Exploration to Storytelling and Back Again , 2016, bioRxiv.

[80]  John B. Black,et al.  Episodes as chunks in narrative memory. , 1979 .

[81]  Ann M. Bisantz,et al.  A comparison of the effects of data–ink ratio on performance with dynamic displays in a monitoring task , 2002 .

[82]  Fernando Pereira,et al.  Automatic video summarization based on MPEG-7 descriptions , 2004, Signal Process. Image Commun..

[83]  Danah Boyd,et al.  Developing legible visualizations for online social spaces , 2002, Proceedings of the 35th Annual Hawaii International Conference on System Sciences.

[84]  Kwan-Liu Ma,et al.  Temporal Summary Images: An Approach to Narrative Visualization via Interactive Annotation Generation and Placement , 2017, IEEE Transactions on Visualization and Computer Graphics.

[85]  Nuno,et al.  Exploratory Study of Urban Flow using Taxi Traces , 2011 .

[86]  Christophe Hurter,et al.  Understanding Data Videos: Looking at Narrative Visualization through the Cinematography Lens , 2015, CHI.

[87]  J. V. van Wijk,et al.  Cluster and calendar based visualization of time series data , 1999, Proceedings 1999 IEEE Symposium on Information Visualization (InfoVis'99).

[88]  Michael Burch,et al.  State-of-the-Art of Visualization for Eye Tracking Data , 2014, EuroVis.

[89]  Ivan Herman,et al.  Graph Visualization and Navigation in Information Visualization: A Survey , 2000, IEEE Trans. Vis. Comput. Graph..

[90]  Deborah Silver,et al.  Dataset traversal with motion-controlled transfer functions , 2005, VIS 05. IEEE Visualization, 2005..

[91]  Victor Kaptelinin,et al.  Window navigation with and without animation: a comparison of scroll bars, zoom, and fisheye view , 1997, CHI Extended Abstracts.

[92]  Guang Li,et al.  SketchComm: a tool to support rich and flexible asynchronous communication of early design ideas , 2012, CSCW.

[93]  Matthew O. Ward,et al.  Interactive Data Visualization - Foundations, Techniques, and Applications , 2010 .

[94]  Till Nagel,et al.  Staged analysis: From evocative to comparative visualizations of Urban mobility , 2017, 2017 IEEE VIS Arts Program (VISAP).

[95]  Ana Figueiras,et al.  Narrative Visualization: A Case Study of How to Incorporate Narrative Elements in Existing Visualizations , 2014, 2014 18th International Conference on Information Visualisation.

[96]  Hanspeter Pfister,et al.  Beyond Memorability: Visualization Recognition and Recall , 2016, IEEE Transactions on Visualization and Computer Graphics.

[97]  Ravi Kumar,et al.  Visualizing tags over time , 2006, WWW '06.

[98]  Judith S. Donath,et al.  Visual Who: animating the affinities and activities of an electronic community , 1995, MULTIMEDIA '95.

[99]  Cláudio T. Silva,et al.  Visual Exploration of Big Spatio-Temporal Urban Data: A Study of New York City Taxi Trips , 2013, IEEE Transactions on Visualization and Computer Graphics.

[100]  Tamara Munzner,et al.  TimeLineCurator: Interactive Authoring of Visual Timelines from Unstructured Text , 2016, IEEE Transactions on Visualization and Computer Graphics.

[101]  Benjamin B. Bederson,et al.  Does animation help users build mental maps of spatial information? , 1999, Proceedings 1999 IEEE Symposium on Information Visualization (InfoVis'99).