Breaking the Fourth Wall of Data Stories through Interaction

Interaction is increasingly integrating into data stories to support data exploration and explanation. Interaction can also be combined with the narrative device, breaking the fourth wall (BTFW), to build a deeper connection between readers and data stories. BTFW interaction directly addresses readers by requiring their input. Such user input is then integrated into the narrative or visuals of data stories to encourage readers to inspect the stories more closely. In this work, we explore the design patterns of BTFW interaction commonly used in data stories. Six design patterns were identified through the analysis of 58 high-quality data stories collected from a range of online sources. Specifically, the data stories were categorized using a coding framework, including the input of BTFW interaction provided by readers and the output of BTFW interaction generated by data stories to respond to the input. To explore the benefits as well as concerns of using BTFW interaction, we conducted a three-session user study including the reading, interview, and recall sessions. The results of our user study suggested that BTFW interaction has a positive impact on self-story connection, user engagement, and information recall. We also discussed design implications to address the possible negative effects on the interactivity-comprehensibility balance, information privacy, and the learning curve of interaction brought by BTFW interaction.

[1]  Yanqiu Wu,et al.  Negative Emotions, Positive Outcomes? Exploring the Communication of Negativity in Serious Data Stories , 2022, CHI.

[2]  Nan Cao,et al.  Understanding Narrative Linearity for Telling Expressive Time-Oriented Stories , 2021, CHI.

[3]  Nan Cao,et al.  Communicating with Motion: A Design Space for Animated Visual Narratives in Data Videos , 2021, CHI.

[4]  Zhicheng Liu,et al.  Leveraging Text-Chart Links to Support Authoring of Data-Driven Articles with VizFlow , 2021, CHI.

[5]  Bongshin Lee,et al.  Data@Hand: Fostering Visual Exploration of Personal Data on Smartphones Leveraging Speech and Touch Interaction , 2021, CHI.

[6]  Jeffrey Heer,et al.  Communicating with Interactive Articles , 2020 .

[7]  Christophe Hurter,et al.  Dear Pictograph: Investigating the Role of Personalization and Immersion for Consuming and Enjoying Visualizations , 2020, CHI.

[8]  Nirmal Kumar Raveendranath,et al.  Pushing the (Visual) Narrative: The Effects of Prior Knowledge Elicitation in Provocative Topics , 2020, CHI.

[9]  Charles Perin,et al.  What is Interaction for Data Visualization? , 2020, IEEE Transactions on Visualization and Computer Graphics.

[10]  Ronald Metoyer,et al.  Linking and Layout: Exploring the Integration of Text and Visualization in Storytelling , 2019, Comput. Graph. Forum.

[11]  Bissie Anderson,et al.  Encoding the UX: User Interface as a Site of Encounter between Data Journalists and Their Constructed Audiences , 2019, Digital Journalism.

[12]  Evan M. Peck,et al.  Data is Personal: Attitudes and Perceptions of Data Visualization in Rural Pennsylvania , 2019, CHI.

[13]  Tobias Höllerer,et al.  XRCreator: interactive construction of immersive data-driven stories , 2018, VRST.

[14]  Frank Biocca,et al.  Exploring immersive experience in journalism , 2018, New Media Soc..

[15]  Cheng Deng,et al.  The Effects of Adding Search Functionality to Interactive Visualizations on the Web , 2018, CHI.

[16]  Yuri Engelhardt,et al.  Narrative Design Patterns for Data-Driven Storytelling , 2018 .

[17]  E. Négrier Breaking the Fourth Wall: proactive Audiences in the Performing Arts , 2018 .

[18]  Bongshin Lee,et al.  Visual Narrative Flow: Exploring Factors Shaping Data Visualization Story Reading Experiences , 2017, Comput. Graph. Forum.

[19]  Katharina Reinecke,et al.  Explaining the Gap: Visualizing One's Predictions Improves Recall and Comprehension of Data , 2017, CHI.

[20]  Bongshin Lee,et al.  Emerging and Recurring Data-Driven Storytelling Techniques: Analysis of a Curated Collection of Recent Stories , 2016 .

[21]  Daniel C. Richardson,et al.  Breaking the Fourth Wall of Cognitive Science , 2016 .

[22]  Jean-Daniel Fekete,et al.  Suggested Interactivity: Seeking Perceived Affordances for Information Visualization , 2016, IEEE Transactions on Visualization and Computer Graphics.

[23]  Märta Kalmaru,et al.  Breaking the fourth wall , 2016 .

[24]  M. Sheelagh T. Carpendale,et al.  More Than Telling a Story: Transforming Data into Visually Shared Stories , 2015, IEEE Computer Graphics and Applications.

[25]  Christophe Hurter,et al.  Understanding Data Videos: Looking at Narrative Visualization through the Cinematography Lens , 2015, CHI.

[26]  Kathleen Langan,et al.  A Dictionary of Literary Terms and Literary Theory: Reference Reviews , 2014 .

[27]  R. Ackoff From Data to Wisdom , 2014 .

[28]  Heidrun Schumann,et al.  A Design Space of Visualization Tasks , 2013, IEEE Transactions on Visualization and Computer Graphics.

[29]  Robert E. Roth,et al.  An Empirically-Derived Taxonomy of Interaction Primitives for Interactive Cartography and Geovisualization , 2013, IEEE Transactions on Visualization and Computer Graphics.

[30]  Tamara Munzner,et al.  A Multi-Level Typology of Abstract Visualization Tasks , 2013, IEEE Transactions on Visualization and Computer Graphics.

[31]  Kamran Sedig,et al.  Transactions on Human-computer Interaction Thci Design for Complex Cognitive Activities with Visual Representations: a Pattern-based Approach Sedig and Parsons Interaction Design for Complex Cognitive Activities with Visualizations , 2022 .

[32]  Neil Cohn,et al.  Visual Narrative Structure , 2013, Cogn. Sci..

[33]  Theresa Horstman,et al.  Transmedia in the classroom: breaking the fourth wall , 2012, MindTrek.

[34]  B. Shneiderman,et al.  Interactive Dynamics for Visual Analysis , 2012 .

[35]  P. Dawson,et al.  ‘Breaking the fourth wall’: 3D virtual worlds as tools for knowledge repatriation in archaeology , 2011 .

[36]  Jeffrey Heer,et al.  Narrative Visualization: Telling Stories with Data , 2010, IEEE Transactions on Visualization and Computer Graphics.

[37]  Ian Bogost,et al.  Newsgames: Journalism at Play , 2010 .

[38]  S. Conway A circular wall? Reformulating the fourth wall for videogames , 2010 .

[39]  G. Freytag Freytag's Technique of the Drama: An Exposition of Dramatic Composition and Art , 2010 .

[40]  Barry Brummett,et al.  Techniques of Close Reading , 2009 .

[41]  Elizabeth Bell,et al.  Theories of Performance , 2008 .

[42]  John T. Stasko,et al.  Toward a Deeper Understanding of the Role of Interaction in Information Visualization , 2007, IEEE Transactions on Visualization and Computer Graphics.

[43]  Jennifer E. Rowley,et al.  The wisdom hierarchy: representations of the DIKW hierarchy , 2007, J. Inf. Sci..

[44]  Imelda Whelehan,et al.  The Cambridge companion to literature on screen , 2007 .

[45]  Marc Cavazza,et al.  Narratology for Interactive Storytelling: A Critical Introduction , 2006, TIDSE.

[46]  V. Braun,et al.  Using thematic analysis in psychology , 2006 .

[47]  Donald A. Norman,et al.  Affordance, conventions, and design , 1999, INTR.

[48]  P. C. Malshe What is this interaction? , 1994, The Journal of the Association of Physicians of India.

[49]  Philip J. Auter Psychometric: TV that talks back: An experimental validation of a parasocial interaction scale , 1992 .

[50]  Philip J. Auter,et al.  When Characters Speak Directly to Viewers: Breaking the Fourth Wall in Television , 1991 .

[51]  J. Nordlund Media Interaction , 1978 .

[52]  Earl A. Knies The “I” of Jane Eyre , 1966, College English.

[53]  Oscar Gross Brockett,et al.  The theatre, an introduction. , 1964 .