Conceptualizing Augmented Reality Television for the Living Room

We examine the concept and characteristics of “Augmented Reality Television” (ARTV) using a four-step investigation method consisting of (1) an analysis of commonly-accepted perspectives on Augmented and Mixed Reality systems, (2) a literature survey of previous work on ARTV, (3) relevant connections with other areas of scientific investigation from TVX/IMX, such as Ambient Media, Interactive TV, and 3-D TV, and (4) by proposing a conceptual framework for ARTV called the “Augmented Reality Television Continuum.” Our work comes at a moment when the excitement and hype about the potential of AR for home entertainment has overlooked rigorous analysis and clear-cut examinations of the concepts involved, which should be the hallmark of any exact science. With this work, our goal is to draw the community’s attention toward fundamentals and first principles of ARTV and to tease out its salient qualities on solid foundations.

[1]  Irina Popovici,et al.  Consolidating the Research Agenda of Augmented Reality Television with Insights from Potential End-Users , 2019, 2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct).

[2]  José Joskowicz,et al.  Experiences on hybrid television and augmented reality on ISDB-T , 2017, 2017 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB).

[3]  Pablo César,et al.  Social television and user interaction , 2008, CIE.

[4]  Ferhan Türe,et al.  Yelling at Your TV: An Analysis of Speech Recognition Errors and Subsequent User Behavior on Entertainment Systems , 2019, SIGIR.

[5]  Levent Onural,et al.  Three-Dimensional Television , 2008 .

[6]  Thomas Risse,et al.  New business, design and models to create semantic ambient media experiences , 2012, Multimedia Tools and Applications.

[7]  David Geerts,et al.  In front of and behind the second screen: viewer and producer perspectives on a companion app , 2014, TVX '14.

[8]  Andrew Jones,et al.  Social TV: toward content navigation using social awareness , 2010 .

[9]  Radu-Daniel Vatavu,et al.  On free-hand TV control: experimental results on user-elicited gestures with Leap Motion , 2015, Personal and Ubiquitous Computing.

[10]  Jun Rekimoto,et al.  ExtVision: Augmentation of Visual Experiences with Generation of Context Images for a Peripheral Vision Using Deep Neural Network , 2018, CHI.

[11]  J. Wobbrock,et al.  Research contributions in human-computer interaction , 2016, Interactions.

[12]  Rhianne Jones,et al.  Object-Based Production: A Personalised Interactive Cooking Application , 2017, TVX.

[13]  David Geerts,et al.  Viewers’ Visions of the Future , 2019, TVX.

[14]  Elizabeth F. Churchill,et al.  Mobile Augmented Reality: A Design Perspective , 2013 .

[15]  Amritpal Singh Bhachu Inclusive technologies for enhancing the accessibility of digital television , 2011, ASAC.

[16]  Matjaz Gams,et al.  Ambient bloom: new business, content, design and models to increase the semantic ambient media experience , 2012, Multimedia Tools and Applications.

[17]  Christian Wolff,et al.  Towards a Comprehensive Definition of Second Screen , 2019, MuC.

[18]  Eyal Ofek,et al.  IllumiRoom: peripheral projected illusions for interactive experiences , 2013, SIGGRAPH '13.

[19]  M. Banusree MIXED REALITY , 2016 .

[20]  Yu Yuan,et al.  All Reality: Virtual, Augmented, Mixed (X), Mediated (X, Y), and Multimediated Reality , 2018, ArXiv.

[21]  Wijnand A. IJsselsteijn,et al.  A survey of perceptual evaluations and requirements of three-dimensional TV , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[22]  Blair MacIntyre,et al.  RoomAlive: magical experiences enabled by scalable, adaptive projector-camera units , 2014, UIST.

[23]  J. Henning Black Mirror: Bandersnatch , 2019 .

[24]  T AzumaRonald Making Augmented Reality a Reality , 2017 .

[25]  Philippe Robert,et al.  Let the sunshine on your screen: introducing augmented reality into interactive television , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[26]  Levent Onural,et al.  Three-Dimensional Television: Capture, Transmission, Display , 2007 .

[27]  Jennifer McNally,et al.  How Millennials and Teens Consume Mobile Video , 2017, TVX.

[28]  Carlos Velasco,et al.  Towards Multisensory Storytelling with Taste and Flavor , 2018, MHFI@ICMI.

[29]  Evropi Stefanidi,et al.  Ambient Intelligence in the Living Room , 2019, Sensors.

[30]  Radu-Daniel Vatavu,et al.  TV Channels in Your Pocket! Linking Smart Pockets to Smart TVs , 2019, TVX.

[31]  Loren G. Terveen,et al.  CollaboraTV: making television viewing social again , 2008, UXTV '08.

[32]  Maxine Glancy,et al.  Personalising the TV Experience using Augmented Reality: An Exploratory Study on Delivering Synchronised Sign Language Interpretation , 2019, CHI.

[33]  Radu-Daniel Vatavu,et al.  Towards Visual Augmentation of the Television Watching Experience: Manifesto and Agenda , 2019, TVX.

[34]  Ana M. Bernardos,et al.  A System to Enable Level-of-Detail Mobile Interaction with Augmented Media Objects , 2014, 2014 Eighth International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing.

[35]  Radu-Daniel Vatavu,et al.  Interactive Coffee Tables: Interfacing TV within an Intuitive, Fun and Shared Experience , 2008, EuroITV.

[36]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[37]  Ronald T. Azuma,et al.  The Most Important Challenge Facing Augmented Reality , 2016, PRESENCE: Teleoperators and Virtual Environments.

[38]  Irina Popovici,et al.  Understanding Users' Preferences for Augmented Reality Television , 2019, International Symposium on Mixed and Augmented Reality.

[39]  Christian Breiteneder,et al.  Virtual Studios: An Overview , 1998, IEEE Multim..

[40]  Radu-Daniel Vatavu,et al.  There's a world outside your TV: exploring interactions beyond the physical TV screen , 2013, EuroITV.

[41]  Manfred Tscheligi,et al.  Tablet, gestures, remote control?: influence of age on performance and user experience with iTV applications , 2014, TVX.

[42]  Hanna Schraffenberger,et al.  Arguably augmented reality: Relationships between the virtual and the real , 2018 .

[43]  Michel Beaudouin-Lafon,et al.  Designing interaction, not interfaces , 2004, AVI.

[44]  Ivan E. Sutherland,et al.  A head-mounted three dimensional display , 1968, AFIPS Fall Joint Computing Conference.

[45]  Stephen A. Brewster,et al.  Examining The Role of Smart TVs and VR HMDs in Synchronous At-a-Distance Media Consumption , 2016, TCHI.

[46]  Rafael Ballagas,et al.  Electric Agents: Combining Collaborative Mobile Augmented Reality and Web-Based Video to Reinvent Interactive Television , 2014, CIE.

[47]  Radu-Daniel Vatavu,et al.  User-defined gestures for free-hand TV control , 2012, EuroITV.

[48]  Albrecht Schmidt,et al.  Matching Information and Ambient Media , 1999, Pers. Ubiquitous Comput..

[49]  Ville Ollikainen,et al.  Mobile TV in Everyday Life Contexts - Individual Entertainment or Shared Experiences? , 2007, EuroITV.

[50]  Kazuhiro Otsuki,et al.  3D free-viewpoint augmented reality on a second display synchronized with a broadcast program , 2018, ICCIP '18.

[51]  Robert Stevens,et al.  Augmenting Television With Augmented Reality , 2019, TVX.

[52]  Ronald T. Azuma,et al.  The road to ubiquitous consumer augmented reality systems , 2019, Human Behavior and Emerging Technologies.

[53]  Radu-Daniel Vatavu,et al.  Leap gestures for TV: insights from an elicitation study , 2014, TVX.

[54]  Michael Evans,et al.  Researching the user experience for connected tv: a case study , 2012, CHI EA '12.

[55]  Philippe Coiffet,et al.  Virtual Reality Technology , 2003, Presence: Teleoperators & Virtual Environments.

[56]  Radu-Daniel Vatavu,et al.  SAPIENS , 2019, Proc. ACM Hum. Comput. Interact..

[57]  Maarten Wijnants,et al.  Interactive Radio Experiences , 2019, TVX.

[58]  Maxine Glancy,et al.  Personalising the TV Experience with Augmented Reality Technology: Synchronised Sign Language Interpretation , 2018, TVX.

[59]  A. James 2010 , 2011, Philo of Alexandria: an Annotated Bibliography 2007-2016.

[60]  Rajiv Ramdhany,et al.  Enabling Frame-Accurate Synchronised Companion Screen Experiences , 2016, TVX.

[61]  Paul Milgram,et al.  A Taxonomy of Real and Virtual World Display Integration , 1999 .

[62]  Ramesh Raskar,et al.  Modern approaches to augmented reality: introduction to current approaches , 2006, SIGGRAPH Courses.

[63]  Fumio Kishino,et al.  Augmented reality: a class of displays on the reality-virtuality continuum , 1995, Other Conferences.

[64]  Joseph A. Paradiso,et al.  Guest Editors' Introduction: Cross-Reality Environments , 2009, IEEE Pervasive Comput..

[65]  Jie Li,et al.  Designing an Object-based Preproduction Tool for Multiscreen TV Viewing , 2018, CHI Extended Abstracts.

[66]  Gary Anthes Augmented reality gets real , 2019, Commun. ACM.

[67]  Christian Geiger,et al.  War Children: Using AR in a Documentary Context , 2018, 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct).

[68]  Ronald Azuma,et al.  A survey of augmented reality" Presence: Teleoperators and virtual environments , 1997 .

[69]  Staffan Björk,et al.  Amplifying Reality , 1999, HUC.

[70]  Oscar Pastor,et al.  A Newcomer's Guide to EICS, the Engineering Interactive Computing Systems Community , 2019, Proc. ACM Hum. Comput. Interact..

[71]  P. Milgram,et al.  A Taxonomy of Mixed Reality Visual Displays , 1994 .

[72]  Pierrick Jouet,et al.  Multi-device mixed reality TV: a collaborative experience with joint use of a tablet and a headset , 2017, VRST.

[73]  Toshio Nakagawa,et al.  Augmented TV: An augmented reality system for TV programs beyond the TV screen , 2014, 2014 International Conference on Multimedia Computing and Systems (ICMCS).

[74]  David Geerts,et al.  Supporting the social uses of television: sociability heuristics for social tv , 2009, CHI.

[75]  Oliver Grau,et al.  Stereoscopic 3D Sports Content without Stereo Rigs , 2010 .

[76]  Radu-Daniel Vatavu,et al.  Audience Silhouettes: Peripheral Awareness of Synchronous Audience Kinesics for Social Television , 2015, TVX.

[77]  Lyndon J. B. Nixon,et al.  LinkedCulture: Browsing Related Europeana Objects while Watching a Cultural Heritage TV Programme , 2015, PATCH@IUI.

[78]  Radu-Daniel Vatavu Point & click mediated interactions for large home entertainment displays , 2010, Multimedia Tools and Applications.

[79]  Radu-Daniel Vatavu Presence bubbles: supporting and enhancing human-human interaction with ambient media , 2010, Multimedia Tools and Applications.

[80]  Hideo Saito,et al.  Augmented reality for 3D TV using depth camera input , 2010, 2010 16th International Conference on Virtual Systems and Multimedia.

[81]  V. Michael Bove,et al.  Immersive media experiences: immersiveme 2013 workshop at ACM multimedia , 2013, MM '13.

[82]  Chiara Rossitto,et al.  Frictional Realities: Enabling Immersion in Mixed-Reality Performances , 2018, TVX.

[83]  Kei Hoshi,et al.  Human-Experiential Design of Presence in Everyday Blended Reality , 2016, Human–Computer Interaction Series.

[84]  Matjaz Gams,et al.  Semantic ambient media: From ambient advertising to ambient-assisted living , 2011, Multimedia Tools and Applications.

[85]  Patrick Langdon,et al.  Developing accessible TV applications , 2011, ASSETS '11.

[86]  Jon Francombe,et al.  Producing Audio Drama Content for an Array of Orchestrated Personal Devices , 2018 .