ipProjector: Designs and Techniques for Geometry-Based Interactive Applications Using a Portable Projector

We propose an interactive projection system for a virtual studio setup using a single self-contained and portable projection device. The system is named ipProjector, which stands for Interactive Portable Projector. Projection allows special effects of a virtual studio to be seen by live audiences in real time. The portable device supports 360-degree shooting and projecting angles and is easy to be integrated with an existing studio setup. We focus on two fundamental requirements of the system and their implementations. First, nonintrusive projection is performed to ensure that the special effect projections and the environment analysis (for locating the target actors or objects) can be performed simultaneously in real time. Our approach uses Digital Light Processing technology, color wheel analysis, and nearest-neighbor search algorithm. Second, a paired projector-camera system is geometrically calibrated with two alternative setups. The first uses a motion sensor for real-time geometric calibration, and the second uses a beam splitter for scene-independent geometric calibration. Based on a small-scale laboratory setting, experiments were conducted to evaluate the geometric accuracy of the proposed approaches, and an application was built to demonstrate the proposed ipProjector concept. Techniques of special effect rendering are not concerned in this paper.

[1]  David G. Lowe,et al.  Shape indexing using approximate nearest-neighbour search in high-dimensional spaces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Henry Fuchs,et al.  Real-Time Projector Tracking on Complex Geometry Using Ordinary Imagery , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Xiang Cao,et al.  Interacting with dynamically defined information spaces using a handheld projector and a pen , 2006, UIST.

[4]  Markus H. Gross,et al.  Interactive environment-aware display bubbles , 2006, UIST.

[5]  James L. Crowley,et al.  Projecting Rectified Images in an Augmented Environment , 2003 .

[6]  Markus H. Gross,et al.  Embedding imperceptible patterns into projected images for simultaneous acquisition and display , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[7]  Greg Welch,et al.  The office of the future: a unified approach to image-based modeling and spatially immersive displays , 1998, SIGGRAPH.

[8]  Srinivasa G. Narasimhan,et al.  Illustrating motion through DLP photography , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[9]  JeroenvanBaar RameshRaskar,et al.  A Low-Cost Projector Mosaic with Fast Registration , 2002 .

[10]  Masanori Sugimoto,et al.  A semi-automatic realtime calibration technique for a handheld projector , 2007, VRST '07.

[11]  Shree K. Nayar,et al.  A Projector-Camera System with Real-Time Photometric Adaptation for Dynamic Environments , 2005, CVPR.

[12]  Mark Fiala,et al.  Automatic Projector Calibration Using Self-Identifying Patterns , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[13]  Yasushi Yagi,et al.  A Sensor for Simultaneously Capturing Texture and Shape by Projecting Structured Infrared Light , 2007, Sixth International Conference on 3-D Digital Imaging and Modeling (3DIM 2007).

[14]  Pattie Maes,et al.  WUW - wear Ur world: a wearable gestural interface , 2009, CHI Extended Abstracts.

[15]  Rahul Sukthankar,et al.  Smarter Presentations: Exploiting Homography in Camera-Projector Systems , 2001, ICCV.

[16]  Weibin Sun,et al.  Robust checkerboard recognition for efficient nonplanar geometry registration in projector-camera systems , 2008, PROCAMS '08.

[17]  Nicolai Petkov,et al.  Comparison of texture features based on Gabor filters , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[18]  Jong-Il Park,et al.  Content Adaptive Embedding of Complementary Patterns for Nonintrusive Direct-Projected Augmented Reality , 2007, HCI.

[19]  Paul A. Beardsley,et al.  Natural video matting using camera arrays , 2006, ACM Trans. Graph..

[20]  Ramesh Raskar,et al.  Seamless Multi-Projector Display on Curved Screens , 2003, IPT/EGVE.

[21]  Jingyi Yu,et al.  Catadioptric projectors , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Johnny Chung Lee,et al.  Hybrid infrared and visible light projection for location tracking , 2007, UIST.

[23]  K. Hosoi.  Cogame : Manipulation by projection , 2007, SIGGRAPH 2007.

[24]  Paul A. Beardsley,et al.  A Handheld Projector Supported by Computer Vision , 2006, ACCV.

[25]  Xiang Cao,et al.  Multi-user interaction using handheld projectors , 2007, UIST.

[26]  Markus H. Gross,et al.  Scalable 3D video of dynamic scenes , 2005, The Visual Computer.

[27]  Paul A. Beardsley,et al.  A self-correcting projector , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[28]  Shuntaro Yamazaki,et al.  Temporal Dithering of Illumination for Fast Active Vision , 2008, ECCV.

[29]  Oliver Bimber,et al.  Dynamic Adaptation of Projected Imperceptible Codes , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.