There's a world outside your TV: exploring interactions beyond the physical TV screen

We explore interactions in the space surrounding the TV set, and use this space as a canvas to display additional TV content in the form of projected screens and customized controls and widgets. We present implementation details for a prototype that creates a hybrid, physical-projected, augmented TV space with off-the-shelf, low-cost technical equipment. The results of a participatory design study are reported to inform development of interaction techniques for multimedia content that spans the physical-projected TV space. We report a set of commands for twelve frequently-used TV tasks and compile guidelines for designing interactions for augmented TV spaces. With this work, we plan to make TV interface designers aware of the many opportunities offered by the space around the physical TV screen once turned interactive. It is our hope that this very first investigation of the interactive potential of the area surrounding the TV set will encourage new explorations of augmented TV spaces, and will foster the design of new applications and new entertainment experiences for our interactive, smart TV spaces of the future.

[1]  Xiang Cao,et al.  Multi-user interaction using handheld projectors , 2007, UIST.

[2]  Pontus Johansson,et al.  Multimodal dialogue systems for interactive TV applications , 2002, Proceedings. Fourth IEEE International Conference on Multimodal Interfaces.

[3]  William T. Freeman,et al.  Television control by hand gestures , 1994 .

[4]  Crysta J. Metcalf,et al.  FANFEEDS: evaluation of socially generated information feed on second screen as a TV show companion , 2012, EuroITV.

[5]  Jens F. Jensen.  Interactive television: new genres, new format, new content , 2005 .

[6]  Alessio Malizia,et al.  The artificiality of natural user interfaces , 2012, CACM.

[7]  Radu-Daniel Vatavu.  Point & click mediated interactions for large home entertainment displays , 2010, Multimedia Tools and Applications.

[8]  Shahram Izadi,et al.  SideSight: multi-"touch" interaction around small devices , 2008, UIST '08.

[9]  I. Scott MacKenzie,et al.  ISO 9241-9 evaluation of video game controllers , 2009, Graphics Interface.

[10]  Ravin Balakrishnan,et al.  Keepin' it real: pushing the desktop metaphor with physics, piles and the pen , 2006, CHI.

[11]  Darren Leigh,et al.  Under the table interaction , 2006, UIST.

[12]  Donald A. Norman,et al.  Natural user interfaces are not natural , 2010, INTR.

[13]  V. Michael Bove,et al.  Surround Vision - A Hand-Held Screen for Accessing Peripheral Content around a Main Screen , 2011, ICIDS.

[14]  Manfred Tscheligi,et al.  Evaluating performance and acceptance of older adults using freehand gestures for TV menu control , 2012, EuroITV.

[15]  Kirstie Hawkey,et al.  LACOME: a multi-user collaboration system for shared large displays , 2012, CSCW.

[16]  Eyal Ofek,et al.  IllumiRoom: peripheral projected illusions for interactive experiences , 2013, SIGGRAPH '13.

[17]  Gilles Bailly,et al.  Gesture-aware remote controls: guidelines and interaction technique , 2011, ICMI '11.

[18]  Jacob O. Wobbrock,et al.  Bonfire: a nomadic system for hybrid laptop-tabletop interaction , 2009, UIST '09.

[19]  Hrvoje Benko,et al.  Combining multiple depth cameras and projectors for interactions on, above and between surfaces , 2010, UIST.

[20]  Radu-Daniel Vatavu,et al.  Creativity in Interactive TV: Personalize, Share, and Invent Interfaces , 2010, Mobile TV: Customizing Content and Experience.

[21]  Johnny Chung Lee,et al.  Hacking the Nintendo Wii Remote , 2008, IEEE Pervasive Computing.

[22]  Pattie Maes,et al.  WUW - wear Ur world: a wearable gestural interface , 2009, CHI Extended Abstracts.

[23]  Radu-Daniel Vatavu,et al.  User-defined gestures for free-hand TV control , 2012, EuroITV.

[24]  Leon Cruickshank,et al.  Making Interactive tv Easier to Use: Interface Design for a Second Screen Approach , 2007 .

[25]  Ana Cristina G. Cañizares.  Fun Rooms: Home Theaters, Music Studios, Game Rooms, and More , 2006 .

[26]  Manfred Tscheligi,et al.  Trends in the living room and beyond: results from ethnographic studies using creative and playful probing , 2008, CIE.

[27]  Chris Harrison,et al.  OmniTouch: wearable multitouch interaction everywhere , 2011, UIST.

[28]  Chris Harrison,et al.  On-body interaction: armed and dangerous , 2012, TEI.

[29]  Desney S. Tan,et al.  Enhancing input on and above the interactive surface with muscle sensing , 2009, ITS '09.

[30]  Niels Henze,et al.  Gesture recognition with a Wii controller , 2008, TEI.

[31]  Saul Greenberg,et al.  The Continuous Interaction Space: Interaction Techniques Unifying Touch and Gesture on and above a Digital Surface , 2011, INTERACT.

[32]  Meredith Ringel Morris,et al.  User-defined gestures for surface computing , 2009, CHI.

[33]  Joseph J. LaViola,et al.  Breaking the status quo: Improving 3D gesture recognition with spatially convenient input devices , 2010, 2010 IEEE Virtual Reality Conference (VR).

[34]  Otmar Hilliges,et al.  Bringing physics to the surface , 2008, UIST '08.

[35]  Evelien D'heer,et al.  Second screen applications and tablet users: constellation, awareness, experience, and interest , 2012, EuroITV.

[36]  Pablo César,et al.  The Evolution of TV Systems, Content, and Users Toward Interactivity , 2009, Found. Trends Hum. Comput. Interact..

[37]  Michael Rohs,et al.  Dynamic ambient lighting for mobile devices , 2011, UIST '11 Adjunct.

[38]  Sebastian Möller,et al.  I'm home: Defining and evaluating a gesture set for smart-home control , 2011, Int. J. Hum. Comput. Stud..

[39]  Andrew D. Wilson.  PlayAnywhere: a compact interactive tabletop projection-vision system , 2005, UIST.

[40]  Otmar Hilliges,et al.  Steerable augmented reality with the beamatron , 2012, UIST.

[41]  Bruce H. Thomas,et al.  A survey of visual, mixed, and augmented reality gaming , 2012, CIE.

[42]  Pablo César,et al.  Usages of the Secondary Screen in an Interactive Television Environment: Control, Enrich, Share, and Transfer Television Content , 2008, EuroITV.

[43]  Wei You,et al.  Studying vision-based multiple-user interaction with in-home large displays , 2008, HCC '08.

[44]  Patrick Baudisch,et al.  Back-of-device interaction allows creating very small touch devices , 2009, CHI.

[45]  Kori Inkpen Quinn,et al.  DeskJockey: Exploiting Passive Surfaces to Display Peripheral Information , 2007, INTERACT.

[46]  Andreas Butz,et al.  Interactions in the air: adding further depth to interactive tabletops , 2009, UIST '09.

[47]  Richard A. Bolt,et al.  “Put-that-there”: Voice and gesture at the graphics interface , 1980, SIGGRAPH '80.

[48]  Kris Luyten,et al.  I Bet You Look Good on the Wall: Making the Invisible Computer Visible , 2009, AmI.

[49]  Markus H. Gross,et al.  Interactive environment-aware display bubbles , 2006, UIST.

[50]  Shin-Dug Kim,et al.  An Effective Model and Management Scheme of Personal Space for Ubiquitous Computing Applications , 2008, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[51]  Frank Bentley,et al.  Ambient social tv: drawing people into a shared experience , 2008, CHI.