Auto-Calibration for Dynamic Multi-Projection Mapping on Arbitrary Surfaces

The quality of every dynamic multi-projection mapping system is limited by the quality of the projector to tracking device calibration. Common problems with poor calibration result in noticeable artifacts for the user, such as ghosting and seams. In this work we introduce a new, fully automated calibration algorithm that is tailored to reduce these artifacts, based on consumer-grade hardware. We achieve this goal by repurposing a structured-light scanning setup. A structured-light scanner can generate 3D geometry based on a known intrinsic and extrinsic calibration of its components (projector and RGB camera). We revert this process by providing the resulting 3D model to determine the intrinsic and extrinsic parameters of our setup (including those of a variety of tracking systems). Our system matches features and solves for all parameters in a single pass while respecting the lower quality of our sensory input.

[1]  Anselm Grundhöfer,et al.  Robust Geometric Self-Calibration of Generic Multi-Projector Camera Systems , 2017, 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[2]  François Berry,et al.  A New Method for Projector Calibration Based on Visual Servoing , 2009, MVA.

[3]  Kosuke Sato,et al.  Fabricating Diminishable Visual Markers for Geometric Registration in Projection Mapping , 2018, IEEE Transactions on Visualization and Computer Graphics.

[4]  Zhengyou Zhang,et al.  Flexible camera calibration by viewing a plane from unknown orientations , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[5]  Blair MacIntyre,et al.  RoomAlive: magical experiences enabled by scalable, adaptive projector-camera units , 2014, UIST.

[6]  Christoph Resch,et al.  On-Site Semi-Automatic Calibration and Registration of a Projector-Camera System Using Arbitrary Objects with Known Geometry , 2015, IEEE Transactions on Visualization and Computer Graphics.

[7]  Christian Siegl,et al.  Robust Blending and Occlusion Compensation in Dynamic Multi-Projection Mapping , 2017, Eurographics.

[8]  Heinrich Niemann,et al.  Point Set Registration with Integrated Scale Estimation , 2005 .

[9]  Ramesh Raskar,et al.  Fast separation of direct and global components of a scene using high frequency illumination , 2006, ACM Trans. Graph..

[10]  Nanning Zheng,et al.  Scaling iterative closest point algorithm for registration of m-D point sets , 2010, J. Vis. Commun. Image Represent..

[11]  Hafeez Anwar,et al.  Projector calibration for 3D scanning using virtual target images , 2012 .

[12]  Ramesh Raskar,et al.  Dynamic shader lamps : painting on movable objects , 2001, Proceedings IEEE and ACM International Symposium on Augmented Reality.

[13]  Takeo Kanade,et al.  Projector Calibration using Arbitrary Planes and Calibrated Camera , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Francisco José Madrid-Cuevas,et al.  Automatic generation and detection of highly reliable fiducial markers under occlusion , 2014, Pattern Recognit..

[15]  Masatoshi Okutomi,et al.  A user-friendly method to geometrically calibrate projector-camera systems , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[16]  Gabriel Taubin,et al.  Simple, Accurate, and Robust Projector-Camera Calibration , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[17]  Tim Weyrich,et al.  A practical structured light acquisition system for point-based geometry and texture , 2005, Proceedings Eurographics/IEEE VGTC Symposium Point-Based Graphics, 2005..

[18]  S. Bougnoux,et al.  From projective to Euclidean space under any practical situation, a criticism of self-calibration , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[19]  Justus Thies,et al.  FaceForge: Markerless Non-Rigid Face Multi-Projection Mapping , 2017, IEEE Transactions on Visualization and Computer Graphics.

[20]  Joni-Kristian Kämäräinen,et al.  Projector Calibration by "Inverse Camera Calibration" , 2011, SCIA.

[21]  Peter F. Sturm,et al.  Geometric video projector auto-calibration , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[22]  Lilong Cai,et al.  A calibration method for uncoupling projector and camera of a structured light system , 2008, 2008 IEEE/ASME International Conference on Advanced Intelligent Mechatronics.