Light Structure from Pin Motion: Geometric Point Light Source Calibration

We present a method for geometric point light source calibration. Unlike prior works that use Lambertian spheres, mirror spheres, or mirror planes, we use a calibration target consisting of a plane and small shadow casters at unknown positions above the plane. We show that shadow observations from a moving calibration target under a fixed light follow the principles of pinhole camera geometry and epipolar geometry, allowing joint recovery of the light position and 3D shadow caster positions, equivalent to how conventional structure from motion jointly recovers camera parameters and 3D feature positions from observed 2D features. Moreover, we devised a unified light model that works with nearby point lights as well as distant light in one common framework. Our evaluation shows that our method yields light estimates that are stable and more accurate than existing techniques while having a much simpler setup and requiring less manual labor.

[1]  Viorica Pătrăucean,et al.  Joint A Contrario Ellipse and Line Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Yang Wang,et al.  Estimation of multiple directional light sources for synthesis of mixed reality images , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[3]  Alexandru Tupan,et al.  Triangulation , 1997, Comput. Vis. Image Underst..

[4]  Dima Damen,et al.  Recognizing linked events: Searching the space of feasible explanations , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Francisco José Madrid-Cuevas,et al.  Automatic generation and detection of highly reliable fiducial markers under occlusion , 2014, Pattern Recognit..

[6]  Richard I. Hartley,et al.  In Defense of the Eight-Point Algorithm , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[8]  Zhao Song,et al.  Photometric stereo with quasi-point light source , 2018, Optics and Lasers in Engineering.

[9]  Nopporn Chotikakamthorn,et al.  Light source estimation using feature points from specular highlights and cast shadows , 2016 .

[10]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[11]  Kaare Brandt Petersen,et al.  The Matrix Cookbook , 2006 .

[12]  Roberto Cipolla,et al.  Semi-Calibrated Near Field Photometric Stereo , 2017, CVPR 2017.

[13]  Katsushi Ikeuchi,et al.  Light source position and reflectance estimation from a single view without the distant illumination assumption , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Hui-Liang Shen,et al.  Calibrating light sources by using a planar mirror , 2011, J. Electronic Imaging.

[15]  Steven M. Seitz,et al.  Shape and spatially-varying BRDFs from photometric stereo , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[16]  David J. Kriegman,et al.  Resolving the Generalized Bas-Relief Ambiguity by Entropy Minimization , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Yasuyuki Matsushita,et al.  Self-calibrating photometric stereo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Atsuto Maki,et al.  Difference sphere: an approach to near light source estimation , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[19]  Xin Pei,et al.  Calibration of position and orientation for point light source synchronously with single image in photometric stereo. , 2019, Optics express.

[20]  Yasuhiro Mukaigawa,et al.  Position estimation of near point light sources using a clear hollow sphere , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[21]  Daniel Cremers,et al.  A Non-convex Variational Approach to Photometric Stereo under Inaccurate Lighting , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Zhenwen Dai,et al.  Polygonal Light Source Estimation , 2009, ACCV.

[23]  Katsushi Ikeuchi,et al.  Illumination from Shadows , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Dmitry B. Goldgof,et al.  A Simple Strategy for Calibrating the Geometry of Light Sources , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Randal C. Nelson,et al.  The Geometry of Point Light Source from Shadows , 2004 .

[26]  In So Kweon,et al.  Semi-Calibrated Photometric Stereo , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Pietro Perona,et al.  3D Photography Using Shadows in Dual-Space Geometry , 1999, International Journal of Computer Vision.

[28]  Jie Wei,et al.  Robust recovery of multiple light source based on local light source constant constraint , 2003, Pattern Recognit. Lett..

[29]  Atsuto Maki,et al.  Difference Sphere: An Approach to Near Light Source Estimation , 2004, CVPR.

[30]  Ersin Yumer,et al.  Learning to predict indoor illumination from a single image , 2017, ACM Trans. Graph..

[31]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[32]  Adrien Bartoli,et al.  3D Reconstruction in Laparoscopy with Close-Range Photometric Stereo , 2012, MICCAI.

[33]  Katsushi Ikeuchi,et al.  Stability issues in recovering illumination distribution from brightness in shadows , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[34]  Berthold K. P. Horn SHAPE FROM SHADING: A METHOD FOR OBTAINING THE SHAPE OF A SMOOTH OPAQUE OBJECT FROM ONE VIEW , 1970 .

[35]  Yee-Hong Yang,et al.  Multiple Illuminant Direction Detection with Application to Image Synthesis , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Xiaochun Cao,et al.  Camera calibration and light source estimation from images with shadows , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[37]  Wei Zhou,et al.  Estimation of Illuminant Direction and Intensity of Multiple Light Sources , 2002, ECCV.

[38]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[39]  Berthold K. P. Horn,et al.  Determining Shape and Reflectance Using Multiple Images , 1978 .

[40]  Kwan-Yee Kenneth Wong,et al.  Camera and light calibration from reflections on a sphere , 2013, Comput. Vis. Image Underst..

[41]  Daniel Cremers,et al.  LED-Based Photometric Stereo: Modeling, Calibration and Numerical Solution , 2017, Journal of Mathematical Imaging and Vision.

[42]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[43]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[44]  Yasuyuki Matsushita,et al.  Self-Calibrating Deep Photometric Stereo Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Kwan-Yee Kenneth Wong,et al.  Recovering Light Directions and Camera Poses from a Single Sphere , 2008, ECCV.

[46]  Roberto Cipolla,et al.  A Practical Method for Estimation of Point Light-Sources , 2001, BMVC.

[47]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[48]  Yasuyuki Matsushita,et al.  Calibrating a Non-isotropic Near Point Light Source Using a Plane , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Roberto Cipolla,et al.  Semi-Calibrated Near Field Photometric Stereo , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  S. Negahdaripour Closed-form relationship between the two interpretations of a moving plane , 1990 .

[51]  Simon Fuhrmann,et al.  Geometric Point Light Source Calibration , 2013, VMV.