Storytelling and Visualization: A Survey

Throughout history, storytelling has been an effective way of conveying information and knowledge. In the field of visualization, storytelling is rapidly gaining momentum and evolving cutting-edge techniques that enhance understanding. Many communities have commented on the importance of storytelling in data visualization. Storytellers tend to be integrating complex visualizations into their narratives in growing numbers. In this paper, we present a survey of storytelling literature in visualization and present an overview of the common and important elements in storytelling visualization. We also describe the challenges in this field as well as a novel classification of the literature on storytelling in visualization. Our classification scheme highlights the open and unsolved problems in this field as well as the more mature storytelling sub-fields. The benefits offer a concise overview and a starting point into this rapidly evolving research trend and provide a deeper understanding of this topic.

[1]  Wei-Ta Chu,et al.  Optimized Comics-Based Storytelling for Temporal Image Sequences , 2015, IEEE Transactions on Multimedia.

[2]  Michelle A. Borkin,et al.  What Makes a Visualization Memorable? , 2013, IEEE Transactions on Visualization and Computer Graphics.

[3]  Ana Figueiras,et al.  Narrative Visualization: A Case Study of How to Incorporate Narrative Elements in Existing Visualizations , 2014, 2014 18th International Conference on Information Visualisation.

[4]  Deborah Silver,et al.  Dataset traversal with motion-controlled transfer functions , 2005, VIS 05. IEEE Visualization, 2005..

[5]  Mina Akaishi,et al.  Narrative based Topic Visualization for Chronological Data , 2007, 2007 11th International Conference Information Visualization (IV '07).

[6]  A. D. Cropper,et al.  Scientific Storytelling: From up in the clouds to down to earth… A new approach to mentoring , 2015, 2015 IEEE Integrated STEM Education Conference.

[7]  Barbara Tversky,et al.  Animation: can it facilitate? , 2002, Int. J. Hum. Comput. Stud..

[8]  Elaine Cohen,et al.  A non-photorealistic lighting model for automatic technical illustration , 1998, SIGGRAPH.

[9]  Jack Zipes,et al.  Creative Storytelling: Building Community/Changing Lives , 1995 .

[10]  Hanspeter Pfister,et al.  Beyond Memorability: Visualization Recognition and Recall , 2016, IEEE Transactions on Visualization and Computer Graphics.

[11]  Victor Kaptelinin,et al.  Window navigation with and without animation: a comparison of scroll bars, zoom, and fisheye view , 1997, CHI Extended Abstracts.

[12]  Guang Li,et al.  SketchComm: a tool to support rich and flexible asynchronous communication of early design ideas , 2012, CSCW.

[13]  Linda B. Gambrell,et al.  Mental imagery, text illustrations, and children's story comprehension and recall. , 1993 .

[14]  Ivan Viola,et al.  Geological storytelling , 2013, Comput. Graph..

[15]  Danah Boyd,et al.  Digital artifacts for remembering and storytelling: posthistory and social network fragments , 2004, 37th Annual Hawaii International Conference on System Sciences, 2004. Proceedings of the.

[16]  Kwan-Liu Ma,et al.  Scientific Storytelling Using Visualization , 2012, IEEE Computer Graphics and Applications.

[17]  P. Pirolli,et al.  The Sensemaking Process and Leverage Points for Analyst Technology as Identified Through Cognitive Task Analysis , 2007 .

[18]  Kwan-Liu Ma,et al.  Design Considerations for Optimizing Storyline Visualizations , 2012, IEEE Transactions on Visualization and Computer Graphics.

[19]  Rosemary Luckin,et al.  Designing multimedia for learning: narrative guidance and narrative construction , 1999, CHI '99.

[20]  Zhen Wen,et al.  Behavior-driven visualization recommendation , 2009, IUI.

[21]  Nahum Gershon,et al.  What storytelling can do for information visualization , 2001, Commun. ACM.

[22]  Jesse Schell,et al.  The Art of Game Design: A book of lenses , 2019 .

[23]  B. L. William Wong,et al.  SchemaLine: Timeline Visualization for Sensemaking , 2014, 2014 18th International Conference on Information Visualisation.

[24]  Edward R. Tufte,et al.  Envisioning Information , 1990 .

[25]  Robert Harper,et al.  Stories in GeoTime , 2007 .

[26]  Adrian Kuhn,et al.  CodeTimeline: Storytelling with versioning data , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[27]  Nicholas Diakopoulos,et al.  Contextifier: automatic generation of annotated stock visualizations , 2013, CHI.

[28]  Johannah Rodgers,et al.  Defining and Experiencing Authorship(s) in the Composition Classroom: Findings from a Qualitative Study of Undergraduate Writing Students at the City University of New York , 2011 .

[29]  John T. Stasko,et al.  Effectiveness of Animation in Trend Visualization , 2008, IEEE Transactions on Visualization and Computer Graphics.

[30]  Nigel Holmes Designer's Guide to Creating Charts and Diagrams , 1984 .

[31]  Jeffrey Heer,et al.  Animated Transitions in Statistical Data Graphics , 2007, IEEE Transactions on Visualization and Computer Graphics.

[32]  Bongshin Lee,et al.  SketchStory: Telling More Engaging Stories with Data through Freeform Sketching , 2013, IEEE Transactions on Visualization and Computer Graphics.

[33]  Ana Figueiras,et al.  How to Tell Stories Using Visualization , 2014, 2014 18th International Conference on Information Visualisation.

[34]  Jock D. Mackinlay,et al.  Storytelling: The Next Step for Visualization , 2013, Computer.

[35]  Bongshin Lee,et al.  Authoring Data-Driven Videos with DataClips , 2017, IEEE Transactions on Visualization and Computer Graphics.

[36]  Bongshin Lee,et al.  A Deeper Understanding of Sequence in Narrative Visualization , 2013, IEEE Transactions on Visualization and Computer Graphics.

[37]  Eser Kandogan,et al.  Just-in-time annotation of clusters, outliers, and trends in point-based data visualizations , 2012, 2012 IEEE Conference on Visual Analytics Science and Technology (VAST).

[38]  Kyle Wm. Hall,et al.  Telling Stories about Dynamic Networks with Graph Comics , 2016, CHI.

[39]  Kwan-Liu Ma,et al.  code_swarm: A Design Study in Organic Software Visualization , 2009, IEEE Transactions on Visualization and Computer Graphics.

[40]  Kwan-Liu Ma,et al.  Software evolution storylines , 2010, SOFTVIS '10.

[41]  Sung-Hee Kim,et al.  Towards a Taxonomy for Evaluating User Engagement in Information Visualization , 2015 .

[42]  David Jensen,et al.  TimeMines: Constructing Timelines with Statistical Models of Word Usage , 2000, KDD 2000.

[43]  George G. Robertson,et al.  Narratives: A visualization to track narrative events as they develop , 2008, 2008 IEEE Symposium on Visual Analytics Science and Technology.

[44]  Brent J. Hecht,et al.  NewsViews: an automated pipeline for creating custom geovisualizations for news , 2014, CHI.

[45]  Jean-Daniel Fekete,et al.  Can Initial Narrative VisualizationTechniques and Storytelling help Engage Online-Users with Exploratory Information Visualizations? , 2015 .

[46]  Yizhou Yu,et al.  Anomaly detection in GPS data based on visual analytics , 2010, 2010 IEEE Symposium on Visual Analytics Science and Technology.

[47]  Benoit Huet,et al.  Automatic video summarization , 2006 .

[48]  Han-Wei Shen,et al.  Interactive Storyboard for Overall Time-Varying Data Visualization , 2008, 2008 IEEE Pacific Visualization Symposium.

[49]  Ivan Viola,et al.  Geological storytelling: graphically exploring and communicating geological sketches , 2012, SBIM '12.

[50]  Paulo Nazareno Maia Sampaio,et al.  MOGRE-Storytelling: Interactive Creation of 3D Stories , 2014, 2014 XVI Symposium on Virtual and Augmented Reality.

[51]  Desney S. Tan,et al.  Phosphor: explaining transitions in the user interface using afterglow effects , 2006, UIST.

[52]  Christophe Hurter,et al.  Understanding Data Videos: Looking at Narrative Visualization through the Cinematography Lens , 2015, CHI.

[53]  J. V. van Wijk,et al.  Cluster and calendar based visualization of time series data , 1999, Proceedings 1999 IEEE Symposium on Information Visualization (InfoVis'99).

[54]  Michael Burch,et al.  State-of-the-Art of Visualization for Eye Tracking Data , 2014, EuroVis.

[55]  Ivan Herman,et al.  Graph Visualization and Navigation in Information Visualization: A Survey , 2000, IEEE Trans. Vis. Comput. Graph..

[56]  Kwan-Liu Ma,et al.  Dynamic video narratives , 2010, SIGGRAPH 2010.

[57]  S. Phithakkitnukoon,et al.  Urban mobility study using taxi traces , 2011, TDMA '11.

[58]  Paula Alavesa,et al.  Combining Storytelling Tradition and Pervasive Gaming , 2013, 2013 5th International Conference on Games and Virtual Worlds for Serious Applications (VS-GAMES).

[59]  Stephen G. Kobourov,et al.  Map‐based Visualizations Increase Recall Accuracy of Data , 2015, Comput. Graph. Forum.

[60]  Martin Wattenberg,et al.  Voyagers and voyeurs: supporting asynchronous collaborative information visualization , 2007, CHI.

[61]  Penousal Machado,et al.  Generative Storytelling for Information Visualization , 2011, IEEE Computer Graphics and Applications.

[62]  Alexander Lex,et al.  From Visual Exploration to Storytelling and Back Again , 2016, bioRxiv.

[63]  Cleotilde Gonzalez,et al.  Does animation in user interfaces improve decision making? , 1996, CHI.

[64]  George M. Kasper,et al.  Animation in user interface design for decision-making: a research framework and empirical analysis , 1995 .

[65]  Mikael Jern,et al.  Geovisual Analytics and Storytelling Using HTML5 , 2013, 2013 17th International Conference on Information Visualisation.

[66]  Lukas Mroz,et al.  RTVR-a flexible Java library for interactive volume rendering , 2001, Proceedings Visualization, 2001. VIS '01..

[67]  John B. Black,et al.  Episodes as chunks in narrative memory. , 1979 .

[68]  Ann M. Bisantz,et al.  A comparison of the effects of data–ink ratio on performance with dynamic displays in a monitoring task , 2002 .

[69]  Ivan Viola,et al.  Importance-Driven Expressive Visualization , 2005 .

[70]  Michael Wohlfart,et al.  Story Telling Aspects in Medical Applications , 2006 .

[71]  Kwan-Liu Ma,et al.  Storytelling via Navigation: A Novel Approach to Animation for Scientific Visualization , 2014, Smart Graphics.

[72]  Jeffrey Heer,et al.  Narrative Visualization: Telling Stories with Data , 2010, IEEE Transactions on Visualization and Computer Graphics.

[73]  Benjamin S. Bloom,et al.  Taxonomy of Educational Objectives: The Classification of Educational Goals. , 1957 .

[74]  Charles Hansen,et al.  The Visualization Handbook , 2011 .

[75]  Carl Gutwin,et al.  Useful junk?: the effects of visual embellishment on comprehension and memorability of charts , 2010, CHI.

[76]  Helwig Hauser,et al.  Story Telling for Presentation in Volume Visualization , 2007, EuroVis.

[77]  Mengchen Liu,et al.  StoryFlow: Tracking the Evolution of Stories , 2013, IEEE Transactions on Visualization and Computer Graphics.

[78]  Andrew Begel,et al.  Codebook: discovering and exploiting relationships in software repositories , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[79]  Antonio Torralba,et al.  Understanding the Intrinsic Memorability of Images , 2011, NIPS.

[80]  Nelson L. Max,et al.  A contract based system for large data visualization , 2005, VIS 05. IEEE Visualization, 2005..

[81]  Kwan-Liu Ma,et al.  AniViz: A Template-Based Animation Tool for Volume Visualization , 2010, IEEE Computer Graphics and Applications.

[82]  Ravi Kumar,et al.  Visualizing tags over time , 2006, WWW '06.

[83]  Judith S. Donath,et al.  Visual Who: animating the affinities and activities of an electronic community , 1995, MULTIMEDIA '95.

[84]  Cláudio T. Silva,et al.  Visual Exploration of Big Spatio-Temporal Urban Data: A Study of New York City Taxi Trips , 2013, IEEE Transactions on Visualization and Computer Graphics.

[85]  Tamara Munzner,et al.  TimeLineCurator: Interactive Authoring of Visual Timelines from Unstructured Text , 2016, IEEE Transactions on Visualization and Computer Graphics.

[86]  Benjamin B. Bederson,et al.  Does animation help users build mental maps of spatial information? , 1999, Proceedings 1999 IEEE Symposium on Information Visualization (InfoVis'99).

[87]  Danah Boyd,et al.  Developing legible visualizations for online social spaces , 2002, Proceedings of the 35th Annual Hawaii International Conference on System Sciences.

[88]  Kwan-Liu Ma,et al.  Temporal Summary Images: An Approach to Narrative Visualization via Interactive Annotation Generation and Placement , 2017, IEEE Transactions on Visualization and Computer Graphics.

[89]  Nuno,et al.  Exploratory Study of Urban Flow using Taxi Traces , 2011 .

[90]  Arvind Satyanarayan,et al.  Authoring Narrative Visualizations with Ellipsis , 2014, Comput. Graph. Forum.

[91]  M. Sheelagh T. Carpendale,et al.  More Than Telling a Story: Transforming Data into Visually Shared Stories , 2015, IEEE Computer Graphics and Applications.

[92]  Shi-Min Hu,et al.  Visual storylines: Semantic visualization of movie sequence , 2012, Comput. Graph..

[93]  Nicholas Diakopoulos,et al.  Visualization Rhetoric: Framing Effects in Narrative Visualization , 2011, IEEE Transactions on Visualization and Computer Graphics.

[94]  Dirk Heylen,et al.  Generating expressive speech for storytelling applications , 2006, IEEE Transactions on Audio, Speech, and Language Processing.