Creativity Support in Projection-Based Augmented Environments

A key theme in ubiquitous computing is to create easy-to-use smart environments in which there is a seamless integration of people, user experience, information, and physical reality. The next generation of ultra-portable handheld projectors is paving the way in this direction, as it develops to its pinnacle. When coupled with appropriate tracking technologies, this technology can extend user experience beyond the confines of the device itself to encompass the physical properties of environment. In this paper, we introduce a projection-based augmented reality framework that aims at facilitating the creation of augmented environments where digital content is snapped to objects around the user. We extend previous literature with a projection-oriented framework that leverages on handheld projectors to seamless combine physical interactions with projection-based content snapped to real objects, while solving inherent issues related to use of projective technology. To evaluate the relevance and impact of this study, we discuss a few application scenarios that are enabling this interaction paradigm.

[1]  Johannes Schöning,et al.  Marauders light: replacing the wand with a mobile camera projector unit , 2009, MUM.

[2]  Oliver Bimber,et al.  Dynamic Adaptation of Projected Imperceptible Codes , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[3]  Jong-Il Park,et al.  Invisible marker based augmented reality system , 2005, Visual Communications and Image Processing.

[4]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[5]  S. Shankar Sastry,et al.  An Invitation to 3-D Vision: From Images to Geometric Models , 2003 .

[6]  Johannes Schöning,et al.  Map torchlight: a mobile augmented reality camera projector unit , 2009, CHI Extended Abstracts.

[7]  Gregory D. Abowd,et al.  TrackSense: Infrastructure Free Precise Indoor Positioning Using Projected Patterns , 2007, Pervasive.

[8]  Elizabeth F. Churchill,et al.  Sharing multimedia content with interactive public displays: a case study , 2004, DIS '04.

[9]  Igor N. Nikitin,et al.  Sceptre: an infrared laser tracking system for virtual environments , 2006, VRST '06.

[10]  Chris Harrison,et al.  OmniTouch: wearable multitouch interaction everywhere , 2011, UIST.

[11]  Darren Leigh,et al.  RFIG lamps: interacting with a self-describing world via photosensing wireless tags and projectors , 2004, SIGGRAPH 2004.

[12]  Ivan Poupyrev,et al.  SideBySide: ad-hoc multi-user interaction with handheld projectors , 2011, UIST.

[13]  Stefano Velasco,et al.  'Sublimotion', un viaje a través de los sentidos , 2014 .

[14]  Andrew D. Wilson PlayAnywhere: a compact interactive tabletop projection-vision system , 2005, UIST.

[15]  Vincent Lepetit,et al.  Fast Keypoint Recognition Using Random Ferns , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Xiang Cao,et al.  Handheld projector interaction , 2009 .

[17]  Lisa G. Cowan,et al.  ShadowPuppets: supporting collocated interaction with mobile projector phones using hand shadows , 2011, CHI.

[18]  Pattie Maes,et al.  WUW - wear Ur world: a wearable gestural interface , 2009, CHI Extended Abstracts.

[19]  Hiroshi Ishii,et al.  Emancipated pixels: real-world graphics in the luminous room , 1999, SIGGRAPH.

[20]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[21]  Xiang Cao,et al.  Flashlight jigsaw: an exploratory study of an ad-hoc multi-player game on public displays , 2008, CSCW.

[22]  Andreas Butz,et al.  Interactions in the air: adding further depth to interactive tabletops , 2009, UIST '09.

[23]  Claudio S. Pinhanez The Everywhere Displays Projector: A Device to Create Ubiquitous Graphical Interfaces , 2001, UbiComp.

[24]  Jane Yung-jen Hsu,et al.  Enabling beyond-surface interactions for interactive surface with an invisible projection , 2010, UIST.

[25]  Seth E. Hunter,et al.  PoCoMo: projected collaboration using mobile devices , 2011, Mobile HCI.

[26]  Masanori Sugimoto,et al.  Hotaru: Intuitive Manipulation Techniques for Projected Displays of Mobile Devices , 2005, INTERACT.

[27]  Masanori Sugimoto,et al.  CoGAME: manipulation using a handheld projector , 2007, SIGGRAPH '07.

[28]  Alina Hang,et al.  Projector phone: a study of using mobile phones with integrated projector for interaction with maps , 2008, Mobile HCI.

[29]  Masanori Sugimoto,et al.  A semi-automatic realtime calibration technique for a handheld projector , 2007, VRST '07.

[30]  Hideo Saito,et al.  Focal Pre-Correction of Projected Image for Deblurring Screen Image , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Naokazu Yokoya,et al.  Localization system for large indoor environments using invisible markers , 2008, VRST '08.

[32]  Johnny Chung Lee,et al.  Hybrid infrared and visible light projection for location tracking , 2007, UIST.

[33]  Martin C. Emele,et al.  SPOTLIGHT NAVIGATION: INTERACTION WITH A HANDHELD PROJECTION DEVICE , 2004 .

[34]  Mark Hereld,et al.  Introduction to building projection-based tiled display systems , 2000, IEEE Computer Graphics and Applications.

[35]  Yongtian Wang,et al.  Study on an indoor tracking system with infrared projected markers for large-area applications , 2009, VRCAI '09.

[36]  Steven K. Feiner,et al.  Exploring interaction with a simulated wrist-worn projection display , 2005, Ninth IEEE International Symposium on Wearable Computers (ISWC'05).

[37]  Tomohiro Tanikawa,et al.  Indoor Marker-based Localization Using Coded Seamless Pattern for Interior Decoration , 2007, 2007 IEEE Virtual Reality Conference.

[38]  Ivan Poupyrev,et al.  Motionbeam: a metaphor for character interaction with handheld projectors , 2011, CHI.

[39]  Roland Siegwart,et al.  BRISK: Binary Robust invariant scalable keypoints , 2011, 2011 International Conference on Computer Vision.

[40]  Ben Congleton,et al.  The context, content & community collage: sharing personal digital media in the physical workplace , 2008, CSCW.

[41]  Paul A. Beardsley,et al.  Interaction using a handheld projector , 2005, IEEE Computer Graphics and Applications.

[42]  Jacob O. Wobbrock,et al.  Bonfire: a nomadic system for hybrid laptop-tabletop interaction , 2009, UIST '09.

[43]  Xiang Cao,et al.  Multi-user interaction using handheld projectors , 2007, UIST.