Sticky Projections-A Model-Based Approach to Interactive Shader Lamps Tracking

Shader lamps can augment physical objects with projected virtual replications using a camera-projector system, provided that the physical and virtual object are well registered to each other. Precise registration and tracking has been a cumbersome and intrusive process in the past. In this paper, we present a new method for tracking complex-shaped physical objects interactively. In contrast to previous approaches our system is mobile and makes solely use of the projection of the virtual replication to track the physical object and “stick” the projection to it. Our method consists of two stages, a fast pose initialization based on structured light patterns and a non-intrusive frame-by-frame tracking based on features detected in the projection. During the tracking phase, a radiometrically corrected virtual camera view based on the current pose prediction is rendered and compared to the captured image. Matched features are triangulated providing a sparse set of surface points that is robustly aligned to the virtual model. The alignment transformation serves as an input for the new pose prediction. Detailed experiments including the evaluation of the overlay accuracy show that our approach can accurately and robustly track complex objects at interactive rates.

[1]  Ramesh Raskar,et al.  Cartoon dioramas in motion , 2005, SIGGRAPH Courses.

[2]  Masatoshi Okutomi,et al.  Direct image alignment of projector-camera systems with planar surfaces , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[4]  David Molyneaux,et al.  Cooperatively Augmenting Smart Objects with Projector-Camera Systems , 2006 .

[5]  Sander Oude Elberink,et al.  Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Applications , 2012, Sensors.

[6]  Markus H. Gross,et al.  Embedding imperceptible patterns into projected images for simultaneous acquisition and display , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[7]  Henry Fuchs,et al.  Real-Time Projector Tracking on Complex Geometry Using Ordinary Imagery , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Kok-Lim Low.  Linear Least-Squares Optimization for Point-to-Plane ICP Surface Registration , 2004 .

[9]  Michitaka Hirose,et al.  Projected augmentation - augmented reality using rotatable video projectors , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[10]  J.-Y. Bouguet,et al.  Pyramidal implementation of the lucas kanade feature tracker , 1999 .

[11]  Christoph Resch,et al.  Sticky projections — A new approach to interactive shader lamp tracking , 2014, 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[12]  Greg Welch,et al.  A general approach for closed-loop registration in AR , 2012, 2013 IEEE Virtual Reality (VR).

[13]  Greg Welch,et al.  Shader Lamps: Animating Real Objects With Image-Based Illumination , 2001, Rendering Techniques.

[14]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[15]  Roel Vertegaal,et al.  DisplayObjects: prototyping functional physical interfaces on 3d styrofoam, paper or cardboard models , 2010, TEI '10.

[16]  J. Hebert,et al.  Geometric Calibration of a Structured Light System Using Circular Control Points , 2008 .

[17]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[18]  Paul A. Beardsley,et al.  Natural video matting using camera arrays , 2006, ACM Trans. Graph..

[19]  Aditi Majumder,et al.  Auto-calibration of cylindrical multi-projector systems , 2010, 2010 IEEE Virtual Reality Conference (VR).

[20]  Ramesh Raskar,et al.  Dynamic shader lamps : painting on movable objects , 2001, Proceedings IEEE and ACM International Symposium on Augmented Reality.

[21]  Ruigang Yang,et al.  Automatic and Continuous Projector Display Surface Estimation Using Everyday Imagery , 2001, WSCG.

[22]  Reinhard Koch,et al.  Visualisation Techniques for Using Spatial Augmented Reality in the Design Process of a Car , 2011, Comput. Graph. Forum.

[23]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[24]  Reinhard Koch,et al.  Evaluation of Geometric Registration Methods for Using Spatial Augmented Reality in the Automotive Industry , 2010, VMV.

[25]  John Underkoffler,et al.  A view from the Luminous Room , 1997, Personal Technologies.

[26]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.