A Research Overview of Mobile Projected User Interfaces

In the last few years, the miniaturization of projectors has gained certain momentum. Today, projectors are available that can easily fit into the palm of a hand. Moreover, these devices are even being integrated into mobile phones. Mobile projectors allow users to project digital imagery into physical space virtually anywhere and anytime. The unique characteristics of small-scale projectors open up interesting opportunities for mobile user interface research. This article provides a comprehensive overview of research on mobile projected user interfaces according to three pertinent research directions: (i) nomadic, (ii) handheld, and (iii) tangible projection interfaces. Furthermore, the article outlines future research challenges and indicates trends in neighboring research fields that might foster innovation in the mobile projected user interfaces domain.

[1]  Max Mühlhäuser,et al.  Permulin: collaboration on interactive surfaces with personal in- and output , 2012, CHI Extended Abstracts.

[2]  Desney S. Tan,et al.  Skinput: appropriating the body as an input surface , 2010, CHI.

[3]  Sebastian Boring,et al.  From Focus to Context and Back: Combining Mobile Projectors and Stationary Displays , 2012 .

[4]  Paul A. Beardsley,et al.  RFIG lamps: interacting with a self-describing world via photosensing wireless tags and projectors , 2004, ACM Trans. Graph..

[5]  Xiang Cao,et al.  PicoPet: "Real World" digital pet on a handheld projector , 2011, UIST '11 Adjunct.

[6]  Johannes Schöning,et al.  Map torchlight: a mobile augmented reality camera projector unit , 2009, CHI Extended Abstracts.

[7]  Moshe Mahler,et al.  HideOut: mobile projector interaction with tangible objects and surfaces , 2013, TEI '13.

[8]  Bernt Schiele,et al.  Vision-Based Detection of Mobile Smart Objects , 2008, EuroSSC.

[9]  Saul Greenberg,et al.  Proxemic interaction: designing for a proximity and orientation-aware environment , 2010, ITS '10.

[10]  Gerd Kortuem,et al.  Smart Sensing and Context, Second European Conference, EuroSSC 2007, Kendal, England, UK, October 23-25, 2007, Proceedings , 2007, EuroSSC.

[11]  Zi Ye,et al.  Cobra: flexible displays for mobilegaming scenarios , 2010, CHI EA '10.

[12]  Karl D. D. Willis.  A pre-history of handheld projector-based interaction , 2011, Personal and Ubiquitous Computing.

[13]  Matt Jones,et al.  PicoTales: collaborative authoring of animated stories using handheld projectors , 2012, CSCW.

[14]  Ivan Poupyrev,et al.  Motionbeam: a metaphor for character interaction with handheld projectors , 2011, CHI.

[15]  Antonio Krüger,et al.  Using Mobile Projection to Support Guitar Learning , 2011, Smart Graphics.

[16]  Sriram Subramanian,et al.  Talking about tactile experiences , 2013, CHI.

[17]  John Hardy,et al.  Toolkit support for interactive projected displays , 2012, MUM.

[18]  Hao Tang,et al.  FACT: fine-grained cross-media interaction with documents via a portable hybrid paper-laptop interface , 2010, ACM Multimedia.

[19]  Hans-Werner Gellersen,et al.  Projected interfaces: enabling serendipitous interaction with smart tangible objects , 2009, Tangible and Embedded Interaction.

[20]  Ali Israr,et al.  AIREAL: interactive tactile experiences in free air , 2013, ACM Trans. Graph..

[21]  Raimund Dachselt.  Interacting with Printed Books Using Digital Pens and Smart Mobile Projection , 2011 .

[22]  Chris Harrison,et al.  OmniTouch: wearable multitouch interaction everywhere , 2011, UIST.

[23]  Darren Leigh,et al.  RFIG lamps: interacting with a self-describing world via photosensing wireless tags and projectors , 2004, SIGGRAPH 2004.

[24]  Uwe Hansmann,et al.  Pervasive Computing , 2003 .

[25]  Alina Hang,et al.  Projector phone: a study of using mobile phones with integrated projector for interaction with maps , 2008, Mobile HCI.

[26]  Paul A. Beardsley,et al.  Natural video matting using camera arrays , 2006, ACM Trans. Graph..

[27]  Ivan Poupyrev,et al.  SideBySide: ad-hoc multi-user interaction with handheld projectors , 2011, UIST.

[28]  Nikolaus F. Troje,et al.  Paper windows: interaction techniques for digital paper , 2005, CHI.

[29]  Hans-Werner Gellersen,et al.  Cooperative augmentation of mobile smart objects with projected displays , 2013, TIIS.

[30]  P. Milgram,et al.  A Taxonomy of Mixed Reality Visual Displays , 1994 .

[31]  Pattie Maes,et al.  Flexpad: highly flexible bending interactions for projected handheld displays , 2013, CHI.

[32]  Xiang Cao,et al.  Interacting with dynamically defined information spaces using a handheld projector and a pen , 2006, UIST.

[33]  Pattie Maes,et al.  LuminAR: portable robotic augmented reality interface design and prototype , 2010, UIST '10.

[34]  Matt Jones,et al.  Mobile and personal projection (MP2) , 2011, CHI Extended Abstracts.

[35]  Lisa G. Cowan,et al.  ShadowPuppets: supporting collocated interaction with mobile projector phones using hand shadows , 2011, CHI.

[36]  Nadir Weibel,et al.  Projector phone use: practices and social implications , 2011, Personal and Ubiquitous Computing.

[37]  Max L. Wilson,et al.  RepliCHI: the workshop , 2013, CHI Extended Abstracts.

[38]  Scott R. Klemmer,et al.  Proceedings of the 24th annual ACM symposium adjunct on User interface software and technology , 2011, UIST 2011.

[39]  Max Mühlhäuser,et al.  LightBeam: interacting with augmented real-world objects in pico projections , 2012, MUM.

[40]  Sriram Subramanian,et al.  Steerable projection: exploring alignment in interactive mobile displays , 2011, Personal and Ubiquitous Computing.

[41]  Michael Naimark.  Two Unusual Projection Spaces , 2005, Presence: Teleoperators & Virtual Environments.

[42]  Hans-Werner Gellersen,et al.  Personal Projectors for Pervasive Computing , 2012, IEEE Pervasive Computing.

[43]  Matt Jones,et al.  Pico-ing into the future of mobile projection and contexts , 2011, Personal and Ubiquitous Computing.

[44]  Tovi Grossman,et al.  PenLight: combining a mobile projector and a digital pen for dynamic visual overlay , 2009, CHI.

[45]  Matt Jones,et al.  Pico projectors: firefly or bright future? , 2012, INTR.

[46]  Ivan Poupyrev,et al.  Lumitrack: low cost, high precision, high speed tracking with projected m-sequences , 2013, UIST.

[47]  Jacob O. Wobbrock,et al.  Bonfire: a nomadic system for hybrid laptop-tabletop interaction , 2009, UIST '09.

[48]  Chris Harrison,et al.  On-body interaction: armed and dangerous , 2012, TEI.

[49]  Jürgen Steimle,et al.  Toward Bimanual Interactions with Mobile Projectors on Arbitrary Surfaces , 2011 .

[50]  Hiroshi Ishii,et al.  Emancipated pixels: real-world graphics in the luminous room , 1999, SIGGRAPH.

[51]  Xiang Cao,et al.  Multi-user interaction using handheld projectors , 2007, UIST.

[52]  Pranav Mistry.  ThirdEye: a technique that enables multiple viewers to see different content on a single display screen , 2009, SIGGRAPH ASIA '09.

[53]  Sebastian Boring,et al.  ProjectorKit: easing rapid prototyping of interactive applications for mobile projectors , 2013, MobileHCI '13.

[54]  Carla Simone,et al.  Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work Companion , 2012, CSCW 2012.

[55]  Ken Pfeuffer,et al.  Investigating mid-air pointing interaction for projector phones , 2012, ITS.

[56]  Johannes Schöning,et al.  Theme issue on personal projection , 2011, Personal and Ubiquitous Computing.

[57]  Pattie Maes,et al.  WUW - wear Ur world: a wearable gestural interface , 2009, CHI Extended Abstracts.

[58]  Enrico Rukzio,et al.  Interactive phone call: synchronous remote collaboration and projected interactive surfaces , 2011, ITS '11.

[59]  Tovi Grossman,et al.  MouseLight: bimanual interactions on digital paper using a pen and a spatially-aware mobile projector , 2010, CHI.

[60]  Martin C. Emele,et al.  SPOTLIGHT NAVIGATION: INTERACTION WITH A HANDHELD PROJECTION DEVICE , 2004 .

[61]  Enrico Rukzio,et al.  Penbook: bringing pen+paper interaction to a tablet device to facilitate paper-based workflows in the hospital domain , 2013, ITS.

[62]  Xiang Cao,et al.  Interactive Environment-Aware Handheld Projectors for Pervasive Computing Spaces , 2012, Pervasive.

[63]  Johannes Schöning,et al.  Marauders light: replacing the wand with a mobile camera projector unit , 2009, MUM.

[64]  Hrvoje Benko,et al.  Combining multiple depth cameras and projectors for interactions on, above and between surfaces , 2010, UIST.