Indoor Scene Reconstruction Using Near-Light Photometric Stereo

We propose a novel framework for photometric stereo (PS) under low-light conditions using uncalibrated near-light illumination. It operates on free-form video sequences captured with a minimalistic and affordable setup. We address issues such as albedo variations, shadowing, perspective projections, and camera noise. Our method uses specular spheres detected with a perspective-correcting Hough transform to robustly triangulate light positions in the presence of outliers via a least-squares approach. Furthermore, we propose an iterative reweighting scheme in combination with an $\ell _{p}$ -norm minimizer to robustly solve the calibrated near-light PS problem. In contrast to other approaches, our framework reconstructs depth, albedo (relative to light source intensity), and normals simultaneously and is demonstrated on synthetic and real-world scenes.

[1]  Hanspeter Pfister,et al.  Visibility Subspaces: Uncalibrated Photometric Stereo with Shadows , 2010, ECCV.

[2]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, ACM Trans. Graph..

[3]  Robert A. McLaughlin,et al.  Randomized Hough Transform: Improved ellipse detection with comparison , 1998, Pattern Recognit. Lett..

[4]  Yuji Iwahori,et al.  Reconstructing shape from shading images under point light source illumination , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[5]  K. Gorski,et al.  HEALPix: A Framework for High-Resolution Discretization and Fast Analysis of Data Distributed on the Sphere , 2004, astro-ph/0409513.

[6]  Ira Kemelmacher-Shlizerman,et al.  Photometric Stereo with General, Unknown Lighting , 2006, International Journal of Computer Vision.

[7]  Saburo Tsuji,et al.  Detection of Ellipses by a Modified Hough Transformation , 1978, IEEE Transactions on Computers.

[8]  Andrea Tagliasacchi,et al.  Sparse Iterative Closest Point , 2013, Comput. Graph. Forum.

[9]  Peiran REN,et al.  Image based relighting using neural networks , 2015, ACM Trans. Graph..

[10]  Paul E. Debevec,et al.  Acquiring the reflectance field of a human face , 2000, SIGGRAPH.

[11]  Philip Dutré,et al.  The free-form light stage , 2002, SIGGRAPH '02.

[12]  Björn Stenger,et al.  Color photometric stereo for multicolored surfaces , 2011, 2011 International Conference on Computer Vision.

[13]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[14]  Holger Winnemöller,et al.  Light Waving: Estimating Light Positions From Photographs Alone , 2005, SIGGRAPH '05.

[15]  Alfred M. Bruckstein,et al.  Near Field Photometric Stereo with Point Light Sources , 2014, SIAM J. Imaging Sci..

[16]  Nahum Kiryati,et al.  Photometric stereo under perspective projection , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[17]  Yasuyuki Matsushita,et al.  A hand-held photometric stereo camera for 3-D modeling , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[18]  Kiyoharu Aizawa,et al.  Robust photometric stereo using sparse regression , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  David J. Kriegman,et al.  Photometric stereo with non-parametric and spatially-varying reflectance , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Simon Fuhrmann,et al.  Geometric Point Light Source Calibration , 2013, VMV.

[21]  Yvain Quéau,et al.  Unifying diffuse and specular reflections for the photometric stereo problem , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[22]  Robert Pless,et al.  Heliometric Stereo: Shape from Sun Position , 2012, ECCV.

[23]  David J. Kriegman,et al.  ShadowCuts: Photometric Stereo with Shadows , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Dmitry B. Goldgof,et al.  A Simple Strategy for Calibrating the Geometry of Light Sources , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Kok Cheong Wong,et al.  Ellipse detection based on symmetry , 1999, Pattern Recognit. Lett..

[26]  Andrew Gardner,et al.  A lighting reproduction approach to live-action compositing , 2002, SIGGRAPH.

[27]  Qiang Ji,et al.  A new efficient ellipse detection method , 2002, Object recognition supported by user interaction for service robots.

[28]  Ravi Ramamoorthi,et al.  On Differential Photometric Reconstruction for Unknown, Isotropic BRDFs , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Charlie C. L. Wang,et al.  Photometric stereo with near point lighting: A solution by mesh deformation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Paolo Favaro,et al.  Uncalibrated Near-Light Photometric Stereo , 2014, BMVC.

[31]  David J. Kriegman,et al.  The Bas-Relief Ambiguity , 2004, International Journal of Computer Vision.

[32]  James J. Clark Active photometric stereo , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[33]  Jiuai Sun,et al.  An improved photometric stereo through distance estimation and light vector optimization from diffused maxima region , 2014, Pattern Recognit. Lett..

[34]  M.K.H. Leung,et al.  Ellipse Detection with Hough Transform in One Dimensional Parametric Space , 2007, 2007 IEEE International Conference on Image Processing.

[35]  Yannick Hold-Geoffroy,et al.  x-Hour Outdoor Photometric Stereo , 2015, 2015 International Conference on 3D Vision.

[36]  K. Sphereo : Determining Depth using Two Specular Spheres and a Single Camera , 1988 .

[37]  Dana H. Ballard,et al.  Generalizing the Hough transform to detect arbitrary shapes , 1981, Pattern Recognit..