Understanding 3D Data Videos: From Screens to Virtual Reality

Data storytelling explores how to communicate data insights to the general public engagingly and effectively. It combines the power of data visualizations and storytelling techniques and is popular in various media such as newspapers, interactive websites, and videos. Recently, virtual reality has brought new opportunities to enhance data storytelling with an incomparable sense of immersion. However, there exists a limited understanding of data stories in virtual reality (VR) as they are still in the early stage. In this paper, we investigated the idea of VR data videos by drawing inspiration from popular 3D data videos and studying how to transfer them from screens to VR. We systematically analyzed 100 highly-watched 3D data videos from Youtube and Tiktok channels to derive their design space. We then conducted a user study with 12 participants to explore the effects of four design factors on user experience, including varying camera angles, showing chart overview, animation, and using anchors. Specifically, participants watched 3D data videos in desktop and VR environments. We collected and analyzed their quantitative and qualitative feedback regarding the story’s understandability, memorability, engagement, and emotional effects. Results suggested that data videos in VR were significantly more appreciated than on desktops. We concluded with design implications for future applications and research on VR data videos.

[1]  Jeffrey Heer,et al.  Cinematic Techniques in Narrative Visualization , 2023, ArXiv.

[2]  Mingming Fan,et al.  From ‘Wow’ to ‘Why’: Guidelines for Creating the Opening of a Data Video with Cinematic Styles , 2022, CHI.

[3]  Nan Cao,et al.  Kineticharts: Augmenting Affective Expressiveness of Charts in Data Stories with Animation Design , 2021, IEEE Transactions on Visualization and Computer Graphics.

[4]  Nan Cao,et al.  A Design Space for Applying the Freytag's Pyramid Structure to Data Stories , 2021, IEEE Transactions on Visualization and Computer Graphics.

[5]  Nan Cao,et al.  AutoClips: An Automatic Approach to Video Generation from Data Facts , 2021, Comput. Graph. Forum.

[6]  Nan Cao,et al.  Communicating with Motion: A Design Space for Animated Visual Narratives in Data Videos , 2021, CHI.

[7]  Nan Cao,et al.  Understanding Narrative Linearity for Telling Expressive Time-Oriented Stories , 2021, CHI.

[8]  Nan Cao,et al.  Smile or Scowl? Looking at Infographic Design Through the Affective Lens , 2021, IEEE Transactions on Visualization and Computer Graphics.

[9]  Dominik Moritz,et al.  AI4VIS: Survey on Artificial Intelligence Approaches for Data Visualization , 2021, IEEE Transactions on Visualization and Computer Graphics.

[10]  Barrett Ens,et al.  Maps Around Me , 2020, Proc. ACM Hum. Comput. Interact..

[11]  Peiran Ren,et al.  Narrative Transitions in Data Videos , 2020, 2020 IEEE Visualization Conference (VIS).

[12]  Bongshin Lee,et al.  Data Visceralization: Enabling Deeper Understanding of Data Using Virtual Reality , 2020, IEEE Transactions on Visualization and Computer Graphics.

[13]  B. Jenny,et al.  Shared Surfaces and Spaces: Collaborative Data Visualisation in a Co-located Immersive Environment , 2020, IEEE Transactions on Visualization and Computer Graphics.

[14]  Johanna Beyer,et al.  Embodied Navigation in Immersive Abstract Data Visualization: Is Overview+Detail or Zooming Better for 3D Scatterplots? , 2020, IEEE Transactions on Visualization and Computer Graphics.

[15]  Yingcai Wu,et al.  What Makes a Data-GIF Understandable? , 2020, IEEE Transactions on Visualization and Computer Graphics.

[16]  Per Ola Kristensson,et al.  Pen-based Interaction with Spreadsheets in Mobile Virtual Reality , 2020, 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[17]  Judd D Bradbury,et al.  Documentary narrative visualization: Features and modes of documentary film in narrative visualization , 2020, Inf. Vis..

[18]  Kim Marriott,et al.  Tilt Map: Interactive Transitions Between Choropleth Map, Prism Map and Bar Chart in Immersive Environments , 2020, IEEE Transactions on Visualization and Computer Graphics.

[19]  Eric D. Ragan,et al.  Scene Transitions and Teleportation in Virtual Reality and the Implications for Spatial Awareness and Sickness , 2020, IEEE Transactions on Visualization and Computer Graphics.

[20]  Bongshin Lee,et al.  Canis: A High‐Level Language for Data‐Driven Chart Animations , 2020, Comput. Graph. Forum.

[21]  Arnaud Prouzeau,et al.  Design and Evaluation of Interactive Small Multiples Data Visualisation in Immersive Spaces , 2020, 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[22]  Danielle Albers Szafir,et al.  Graphical Perception for Immersive Analytics , 2020, 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[23]  M. Kraus,et al.  The Impact of Immersion on Cluster Identification Tasks , 2020, IEEE Transactions on Visualization and Computer Graphics.

[24]  Wolfgang Stuerzlinger,et al.  Evaluating an Immersive Space-Time Cube Geovisualization for Intuitive Trajectory Data Exploration , 2019, IEEE Transactions on Visualization and Computer Graphics.

[25]  Arnaud Prouzeau,et al.  Scaptics and Highlight-Planes: Immersive Interaction Techniques for Finding Occluded Features in 3D Scatterplots , 2019, CHI.

[26]  Karrie Karahalios,et al.  Understanding Visual Cues in Visualizations Accompanied by Audio Narrations , 2019, CHI.

[27]  Wesley Willett,et al.  A Walk Among the Data , 2019, IEEE Computer Graphics and Applications.

[28]  Yalong Yang,et al.  Origin-Destination Flow Maps in Immersive Environments , 2019, IEEE Transactions on Visualization and Computer Graphics.

[29]  Tobias Höllerer,et al.  XRCreator: interactive construction of immersive data-driven stories , 2018, VRST.

[30]  Arvind Satyanarayan,et al.  Vega-Lite: A Grammar of Interactive Graphics , 2018, IEEE Transactions on Visualization and Computer Graphics.

[31]  Bongshin Lee,et al.  Hooked on data videos: assessing the effect of animation and pictographs on viewer engagement , 2018, AVI.

[32]  Robert S. Laramee,et al.  Storytelling and Visualization: An Extended Survey , 2018, Inf..

[33]  Bruce H. Thomas,et al.  Combining Virtual Reality and Narrative Visualisation to Persuade , 2017, 2017 International Symposium on Big Data Visual Analytics (BDVA).

[34]  Bongshin Lee,et al.  Timelines Revisited: A Design Space and Considerations for Expressive Storytelling , 2017, IEEE Transactions on Visualization and Computer Graphics.

[35]  Cheng-Hsin Hsu,et al.  360° Video Viewing Dataset in Head-Mounted Virtual Reality , 2017, MMSys.

[36]  Kwan-Liu Ma,et al.  A gesture system for graph visualization in virtual reality environments , 2017, 2017 IEEE Pacific Visualization Symposium (PacificVis).

[37]  Bongshin Lee,et al.  Authoring Data-Driven Videos with DataClips , 2017, IEEE Transactions on Visualization and Computer Graphics.

[38]  Alex Endert,et al.  Beyond Usability and Performance: A Review of User Experience-focused Evaluations in Visualization , 2016, BELIV '16.

[39]  Kwan-Liu Ma,et al.  A Study of Layout, Rendering, and Interaction Methods for Immersive Graph Visualization , 2016, IEEE Transactions on Visualization and Computer Graphics.

[40]  Steven K. Feiner,et al.  Combating VR sickness through subtle dynamic field-of-view modification , 2016, 2016 IEEE Symposium on 3D User Interfaces (3DUI).

[41]  Christophe Hurter,et al.  Understanding Data Videos: Looking at Narrative Visualization through the Cinematography Lens , 2015, CHI.

[42]  Steven Franconeri,et al.  ISOTYPE Visualization: Working Memory, Performance, and Engagement with Pictographs , 2015, CHI.

[43]  Hanspeter Pfister,et al.  What Makes a Visualization Memorable? , 2013, IEEE Transactions on Visualization and Computer Graphics.

[44]  Jeffrey Heer,et al.  Narrative Visualization: Telling Stories with Data , 2010, IEEE Transactions on Visualization and Computer Graphics.

[45]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[46]  Bongshin Lee,et al.  Immersive Visual Data Stories , 2018, Immersive Analytics.

[47]  David Bordwell,et al.  Film Art: An Introduction , 1979 .