Albedo assisted high-quality shape recovery from 4D light fields

Over the past decade, shape reconstruction methods have been limited to Lambertian reflectance with uniform albedo and controlled lighting environment. In this paper, we present an approach for recovering high-quality shapes from 4D light fields, which can handle non-Lambertian and multi-albedo scenes with shadows and inter-reflections. 4D light fields represent all light rays that hit the sensor plane from different directions, and the depth map from light fields is robust to non-Lambertian objects. Specifically, we estimate the albedos by eliminating shadows and inter-reflections with the edge and chromaticity. Then the lighting environment is analyzed from albedos and shading. Finally, the high-quality surface geometry is exactly recovered through normal refinement. We evaluate the effectiveness and robustness on the public 4D light fields database with both synthetic and real-world scenes.

[1]  Venu Madhav Govindu,et al.  High Quality Photometric Reconstruction Using a Depth Camera , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Yasuyuki Matsushita,et al.  Self-calibrating photometric stereo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  Sven Wanner,et al.  Variational Light Field Analysis for Disparity Estimation and Super-Resolution , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Edward H. Adelson,et al.  Shape estimation in natural illumination , 2011, CVPR 2011.

[5]  Stephen Lin,et al.  Shading-Based Shape Refinement of RGB-D Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Qionghai Dai,et al.  Fusing Multiview and Photometric Stereo for 3D Reconstruction under Uncalibrated Illumination , 2011, IEEE Transactions on Visualization and Computer Graphics.

[7]  Yasuyuki Matsushita,et al.  Multiview Photometric Stereo Using Planar Mesh Parameterization , 2013, 2013 IEEE International Conference on Computer Vision.

[8]  Sven Wanner,et al.  Globally consistent depth labeling of 4D light fields , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Szymon Rusinkiewicz,et al.  Efficiently combining positions and normals for precise 3D geometry , 2005, ACM Trans. Graph..

[10]  Sven Wanner,et al.  Datasets and Benchmarks for Densely Sampled 4D Light Fields , 2013, VMV.

[11]  In-So Kweon,et al.  Exploiting Shading Cues in Kinect IR Images for Geometry Refinement , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  In-So Kweon,et al.  High Quality Shape from a Single RGB-D Image under Uncalibrated Natural Illumination , 2013, 2013 IEEE International Conference on Computer Vision.

[13]  Thomas Pock,et al.  Shape from Light Field Meets Robust PCA , 2014, ECCV.

[14]  Jean-Denis Durou,et al.  Numerical methods for shape-from-shading: A new survey with benchmarks , 2008, Comput. Vis. Image Underst..

[15]  Vladlen Koltun,et al.  A Simple Model for Intrinsic Image Decomposition with Depth Cues , 2013, 2013 IEEE International Conference on Computer Vision.

[16]  Sven Wanner,et al.  Globally Consistent Multi-label Assignment on the Ray Space of 4D Light Fields , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.