Directionally Decomposing Structured Light for Projector Calibration

Intrinsic projector calibration is essential in projection mapping (PM) applications, especially in dynamic PM. However, due to the shallow depth-of-field (DOF) of a projector, more work is needed to ensure accurate calibration. We aim to estimate the intrinsic parameters of a projector while avoiding the limitation of shallow DOF. As the core of our technique, we present a practical calibration device that requires a minimal working volume directly in front of the projector lens regardless of the projector's focusing distance and aperture size. The device consists of a flat-bed scanner and pinhole-array masks. For calibration, a projector projects a series of structured light patterns in the device. The pinholes directionally decompose the structured light, and only the projected rays that pass through the pinholes hit the scanner plane. For each pinhole, we extract a ray passing through the optical center of the projector. Consequently, we regard the projector as a pinhole projector that projects the extracted rays only, and we calibrate the projector by applying the standard camera calibration technique, which assumes a pinhole camera model. Using a proof-of-concept prototype, we demonstrate that our technique can calibrate projectors with different focusing distances and aperture sizes at the same accuracy as a conventional method. Finally, we confirm that our technique can provide intrinsic parameters accurate enough for a dynamic PM application, even when a projector is placed too far from a projection target for a conventional method to calibrate the projector using a fiducial object of reasonable size.

[1]  Takeo Kanade,et al.  Simultaneous self-calibration of a projector and a camera using structured light , 2011, CVPR 2011 WORKSHOPS.

[2]  Aditi Majumder,et al.  Autocalibrating Tiled Projectors on Piecewise Smooth Vertically Extruded Surfaces , 2011, IEEE Transactions on Visualization and Computer Graphics.

[3]  Anselm Grundhöfer,et al.  Robust Geometric Self-Calibration of Generic Multi-Projector Camera Systems , 2017, 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[4]  Kosuke Sato,et al.  Document search support by making physical documents transparent in projection-based mixed reality , 2011, Virtual Reality.

[5]  Kosuke Sato,et al.  Projection-based visualization of tangential deformation of nonrigid surface by deformation estimation using infrared texture , 2015, Virtual Reality.

[6]  Masatoshi Ishikawa,et al.  Dynamic Projection Mapping onto Deforming Non-Rigid Surface Using Deformable Dot Cluster Marker , 2017, IEEE Transactions on Visualization and Computer Graphics.

[7]  Takeo Kanade,et al.  Accurate camera calibration using iterative refinement of control points , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[8]  Justus Thies,et al.  Real-time pixel luminance optimization for dynamic multi-projection mapping , 2015, ACM Trans. Graph..

[9]  Lina J. Karam,et al.  Camera calibration using adaptive segmentation and ellipse fitting for localizing control points , 2012, 2012 19th IEEE International Conference on Image Processing.

[10]  Kosuke Sato,et al.  Fabricating Diminishable Visual Markers for Geometric Registration in Projection Mapping , 2018, IEEE Transactions on Visualization and Computer Graphics.

[11]  Kosuke Sato,et al.  Material Surface Reproduction and Perceptual Deformation with Projection Mapping for Car Interior Design , 2019, 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[12]  Steve Marschner,et al.  Dual photography , 2005, ACM Trans. Graph..

[13]  Kosuke Sato,et al.  Optical superimposition of infrared thermography through video projection , 2010 .

[14]  Masatoshi Ishikawa,et al.  Lumipen 2: Dynamic Projection Mapping with Mirror-Based Robust High-Speed Tracking against Illumination Changes , 2016, PRESENCE: Teleoperators and Virtual Environments.

[15]  Masatoshi Okutomi,et al.  A user-friendly method to geometrically calibrate projector-camera systems , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[16]  Kosuke Sato,et al.  Interactive bookshelf surface for in situ book searching and storing support , 2011, AH '11.

[17]  Peisen S. Huang,et al.  Novel method for structured light system calibration , 2006 .

[18]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Toshiyuki Amano,et al.  Projection Center Calibration for a Co-located Projector Camera System , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[20]  Justus Thies,et al.  FaceForge: Markerless Non-Rigid Face Multi-Projection Mapping , 2017, IEEE Transactions on Visualization and Computer Graphics.

[21]  Gabriel Taubin,et al.  Simple, Accurate, and Robust Projector-Camera Calibration , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[22]  Tomoyuki Saito,et al.  Real-time Navigation for Liver Surgery Using Projection Mapping With Indocyanine Green Fluorescence: Development of the Novel Medical Imaging Projection System , 2017, Annals of surgery.

[23]  Toshiyuki Amano,et al.  Projection Based Real-Time Material Appearance Manipulation , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[24]  Guillaume Moreau,et al.  Practical and Precise Projector-Camera Calibration , 2016, 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[25]  Anselm Grundhöfer,et al.  Projection-Based Augmented Reality in Disney Theme Parks , 2012, Computer.

[26]  Beiwen Li,et al.  Novel calibration method for structured-light system with an out-of-focus projector. , 2014, Applied optics.

[27]  Ramesh Raskar,et al.  Modern approaches to augmented reality: introduction to current approaches , 2006, SIGGRAPH Courses.

[28]  Kosuke Sato,et al.  Geometrically Consistent Projection-Based Tabletop Sharing for Remote Collaboration , 2018, IEEE Access.

[29]  Anselm Grundhöfer,et al.  Recent Advances in Projection Mapping Algorithms, Hardware and Applications , 2018, Comput. Graph. Forum.

[30]  Thabo Beeler,et al.  A Practical Method for Fully Automatic Intrinsic Camera Calibration Using Directionally Encoded Light , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Anselm Grundhöfer,et al.  Makeup Lamps: Live Augmentation of Human Faces via Projection , 2017, Comput. Graph. Forum.