Cross-device media: a review of second screening and multi-device television

Television viewers interacting with second screens has become a common sight in the modern living room. Such activities are a mixture of related, semi-related, and non-related browsing of content. This growing trend is revolutionising the way that broadcasters think about their content. Through the envisioned connected home, driven by end-to-end IP connected networks, television content creators and app developers are now considering the design space for multi-device, interactive experiences. In this review paper, we consider the pre-digital beginnings of such scenarios and progress to discuss how the introduction of mobile devices has affected the TV viewing experience. We discuss dual-screen usage over a variety of contexts in the connected home, with a focus on ‘designed’ dual-screen experiences such as companion applications. We conclude with reflections on the future of this area so that app developers, broadcasters, and academics may push further the space and improve future dual- and multi-screen experiences.

[1]  J. Sachs.  Memory in reading and listening to discourse , 1974, Memory & cognition.

[2]  Marita Franzke,et al.  Dual device user interface design: PDAs and interactive television , 1996, CHI.

[3]  Hiroshi Ishii,et al.  Ambient Displays: Turning Architectural Space into an Interface between People and Digital Information , 1998, CoBuild.

[4]  James H. Aylor,et al.  Computer for the 21st Century , 1999, Computer.

[5]  Marilyn Tremaine.  CHI '01 Extended Abstracts on Human Factors in Computing Systems , 2001, CHI Extended Abstracts.

[6]  John T. Stasko,et al.  The InfoCanvas: information conveyance through personalized, expressive art , 2001, CHI Extended Abstracts.

[7]  Jonathan Grudin,et al.  Partitioning digital worlds: focal and peripheral awareness in multiple monitor use , 2001, CHI.

[8]  Daniel R. Anderson,et al.  Viewing the viewers: Viewing behaviors by children and adults during television programs and commercials , 2003 .

[9]  Mark T. Maybury,et al.  Personalcasting: Tailored Broadcast News , 2004, User Modeling and User-Adapted Interaction.

[10]  Tim Regan,et al.  Media center buddies: instant messaging around a media center , 2004, NordiCHI '04.

[11]  Judith Masthoff,et al.  A dual device scenario for informal language learning: interactive television meets the mobile phone , 2004, IEEE International Conference on Advanced Learning Technologies, 2004. Proceedings..

[12]  Tara Matthews,et al.  A toolkit for managing user attention in peripheral displays , 2004, UIST '04.

[13]  Michael Rohs,et al.  The smart phone: a ubiquitous input device , 2006, IEEE Pervasive Computing.

[14]  Pirkko Oittinen,et al.  Effects of extra information on TV viewers' visual attention, message processing ability, and cognitive workload , 2007, CIE.

[15]  Leon Cruickshank,et al.  Making Interactive tv Easier to Use: Interface Design for a Second Screen Approach , 2007 .

[16]  Manfred Tscheligi,et al.  Trends in the living room and beyond: results from ethnographic studies using creative and playful probing , 2008, CIE.

[17]  David Dearman,et al.  It's on my other computer!: computing with multiple devices , 2008, CHI.

[18]  Pablo César,et al.  The implications of program genres for the design of social television systems , 2008, UXTV '08.

[19]  Jonathan Rosen,et al.  Distractions, Distractions: Does Instant Messaging Affect College Students' Performance on a Concurrent Reading Comprehension Task? , 2009, Cyberpsychology Behav. Soc. Netw..

[20]  David Geerts,et al.  Supporting the social uses of television: sociability heuristics for social tv , 2009, CHI.

[21]  Emmanuel Tsekleves,et al.  Bringing the television experience to other media in the home: an ethnographic study , 2009, EuroITV '09.

[22]  Konstantinos Chorianopoulos,et al.  Delegating the visual interface between a tablet and a TV , 2010, AVI.

[23]  Jun-Geol Baek,et al.  An integrated music video browsing system for personalized television , 2011, Expert Syst. Appl..

[24]  Volker Wulf,et al.  Jumping between devices and services: towards an integrated concept for social tv , 2011, EuroITV '11.

[25]  Rufael Mekuria,et al.  Are we in sync?: synchronization requirements for watching online video together. , 2011, CHI.

[26]  Crysta J. Metcalf,et al.  Field trial of a dual device user experience for iTV , 2011, EuroITV '11.

[27]  Paul Coulton,et al.  Mobile phones as second screen for TV, enabling inter-audience interaction , 2011, Advances in Computer Entertainment Technology.

[28]  James Gips,et al.  Media Multitasking Behavior: Concurrent Television and Computer Usage , 2011, Cyberpsychology Behav. Soc. Netw..

[29]  Evelien D'heer,et al.  Everyday life in (front of) the screen: the consumption of multiple screen technologies in the living room context , 2012, EuroITV.

[30]  Jordi Carrabina,et al.  Subtitle Synchronization across Multiple Screens and Devices , 2012, Sensors.

[31]  Jennifer Pearson,et al.  Co-reading: investigating collaborative group reading , 2012, JCDL '12.

[32]  Crysta J. Metcalf,et al.  FANFEEDS: evaluation of socially generated information feed on second screen as a TV show companion , 2012, EuroITV.

[33]  Michael E. Holmes,et al.  Visual attention to television programs with a second-screen application , 2012, ETRA '12.

[34]  David Geerts,et al.  Second-Screen Use in the Home: an Ethnographic Study , 2012 .

[35]  Janet H. Murray,et al.  Story-map: iPad companion for long form TV narratives , 2012, EuroITV.

[36]  Evelien D'heer,et al.  Second screen applications and tablet users: constellation, awareness, experience, and interest , 2012, EuroITV.

[37]  Adrian Benton,et al.  Social TV: real-time social media response to TV advertising , 2012, ADKDD '12.

[38]  Michael Evans,et al.  Researching the user experience for connected tv: a case study , 2012, CHI EA '12.

[39]  Jordan L. Boyd-Graber,et al.  Spoiler alert: Machine learning approaches to detect social media posts with revelatory information , 2013, ASIST.

[40]  Gianmarco,et al.  The Future of Second Screen Experience , 2013 .

[41]  Frank Melchior,et al.  Object-based audio applied to football broadcasts , 2013, ImmersiveMe '13.

[42]  Han Cao,et al.  Social TV EPG Interaction Design for Multi-screen Environment , 2013, 2013 IEEE International Conference on Green Computing and Communications and IEEE Internet of Things and IEEE Cyber, Physical and Social Computing.

[43]  Shekhawat Yahoo Labs.  The Future of Second Screen Experience , 2013 .

[44]  Daniel J. Wigdor,et al.  A field study of multi-device workflows in distributed workspaces , 2013, UbiComp.

[45]  Radu-Daniel Vatavu,et al.  There's a world outside your TV: exploring interactions beyond the physical TV screen , 2013, EuroITV.

[46]  Eyal Ofek,et al.  IllumiRoom: peripheral projected illusions for interactive experiences , 2013, SIGGRAPH '13.

[47]  Roberto Pagano,et al.  TV program detection in tweets , 2013, EuroITV.

[48]  Otmar Scherzer,et al.  The effect of cinematic cuts on human attention , 2014, TVX '14.

[49]  Stephen A. Brewster,et al.  Mirror, mirror, on the wall: collaborative screen-mirroring for small groups , 2014, TVX.

[50]  Steve Benford,et al.  Many-screen viewing: evaluating an olympics companion application , 2014, TVX '14.

[51]  Matthew Chalmers,et al.  Practices of parallel media: using mobile devices when watching television , 2014 .

[52]  Andy Brown,et al.  HCI over multiple screens , 2014, CHI Extended Abstracts.

[53]  Daniel J. Wigdor,et al.  Panelrama: enabling easy specification of cross-device web applications , 2014, CHI.

[54]  Martin R. Gibbs,et al.  Screen ecologies, multi-gaming and designing for different registers of engagement , 2014, CHI PLAY.

[55]  Manfred Tscheligi,et al.  Tablet, gestures, remote control?: influence of age on performance and user experience with iTV applications , 2014, TVX.

[56]  Katharina Emmerich,et al.  Defining second screen gaming: exploration of new design patterns , 2014, Advances in Computer Entertainment.

[57]  Ana Tajadura-Jiménez,et al.  Working with the television on: an investigation into media multitasking , 2014, CHI Extended Abstracts.

[58]  Bryan C. Semaan,et al.  Hybrid media consumption: how tweeting during a televised political debate influences the vote decision , 2014, CSCW.

[59]  David Geerts,et al.  In front of and behind the second screen: viewer and producer perspectives on a companion app , 2014, TVX '14.

[60]  Frank Melchior,et al.  Object-based broadcasting - curation, responsiveness and user experience , 2014 .

[61]  Radu-Daniel Vatavu,et al.  Visual attention measures for multi-screen TV , 2014, TVX '14.

[62]  Brown Andy,et al.  Eye-tracking the dual-screen experience , 2014 .

[63]  Frank Bentley,et al.  "I'm just on my phone and they're watching TV": Quantifying mobile device use while watching television , 2015, TVX.

[64]  Julián Flórez,et al.  A Web-based distributed architecture for multi-device adaptation in media applications , 2015, Personal and Ubiquitous Computing.

[65]  Skjalg Lepsøy,et al.  Making Second Screen Sustainable in Media Production: the BRIDGET Approach , 2015, TVX.

[66]  Wendy E. Mackay,et al.  Webstrates: Shareable Dynamic Media , 2015, UIST.

[67]  Michael Evans,et al.  Mediating Attention for Second Screen Companion Content , 2015, CHI.

[68]  Janet Murray,et al.  United Universe: A Second Screen Transmedia Experience , 2015, TVX.

[69]  Stephen A. Brewster,et al.  It Takes Two (To Co-View): Collaborative Multi-View TV , 2015, TVX.

[70]  Matt Jones,et al.  Designing attention for multi-screen TV experiences , 2015, BCS HCI.

[71]  Charlotte Hoare,et al.  Hide and Seek: Exploring Interaction With Smart Wallpaper , 2015, CHI PLAY.

[72]  Hana Kim,et al.  Companion apps for information-rich television programmes: representation and interaction , 2015, Personal and Ubiquitous Computing.

[73]  Frank Honold,et al.  How Companion-Technology can Enhance a Multi-Screen Television Experience: A Test Bed for Adaptive Multimodal Interaction in Domestic Environments , 2016, KI - Künstliche Intelligenz.

[74]  Rajiv Ramdhany,et al.  Enabling Frame-Accurate Synchronised Companion Screen Experiences , 2016, TVX.

[75]  Hans Stokking,et al.  Standards for multi-stream and multi-device media synchronization , 2016, IEEE Communications Magazine.

[76]  Anthony D. Wagner,et al.  Media multitasking and memory: Differences in working memory and long-term memory , 2016, Psychonomic bulletin & review.

[77]  Matt Jones,et al.  Designing Visual Complexity for Dual-screen Media , 2016, CHI.

[78]  Jennifer Pearson,et al.  Emergeables: Deformable Displays for Continuous Eyes-Free Mobile Interaction , 2016, CHI.

[79]  Matt Jones,et al.  Interdevice Media: Choreographing Content to Maximize Viewer Engagement , 2016, Computer.

[80]  M. Patterson,et al.  A Naturalistic Investigation of Media Multitasking While Studying and the Effects on Exam Performance , 2017 .