Shading-Based Shape Refinement of RGB-D Images

We present a shading-based shape refinement algorithm which uses a noisy, incomplete depth map from Kinect to help resolve ambiguities in shape-from-shading. In our framework, the partial depth information is used to overcome bas-relief ambiguity in normals estimation, as well as to assist in recovering relative albedos, which are needed to reliably estimate the lighting environment and to separate shading from albedo. This refinement of surface normals using a noisy depth map leads to high-quality 3D surfaces. The effectiveness of our algorithm is demonstrated through several challenging real-world examples.

[1]  Michael S. Brown,et al.  High quality depth map upsampling for 3D-TOF cameras , 2011, 2011 International Conference on Computer Vision.

[2]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[3]  Tony F. Chan,et al.  Outdoor photometric stereo , 2013, IEEE International Conference on Computational Photography (ICCP).

[4]  Shree K. Nayar,et al.  Reflectance based object recognition , 1996, International Journal of Computer Vision.

[5]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[6]  Michael S. Brown,et al.  A framework for ultra high resolution 3D imaging , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Ko Nishino,et al.  Shape and Reflectance from Natural Illumination , 2012, ECCV.

[8]  Sebastian Thrun,et al.  Upsampling range data in dynamic environments , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Pat Hanrahan,et al.  An efficient representation for irradiance environment maps , 2001, SIGGRAPH.

[10]  Edward H. Adelson,et al.  Shape estimation in natural illumination , 2011, CVPR 2011.

[11]  Brian V. Funt,et al.  Color Constant Color Indexing , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[13]  Yasuyuki Matsushita,et al.  High-quality shape from multi-view stereo and shading under general illumination , 2011, CVPR 2011.

[14]  Ruigang Yang,et al.  Spatial-Depth Super Resolution for Range Images , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Olivier D. Faugeras,et al.  Shape From Shading , 2006, Handbook of Mathematical Models in Computer Vision.

[16]  Jean-Denis Durou,et al.  Numerical methods for shape-from-shading: A new survey with benchmarks , 2008, Comput. Vis. Image Underst..

[17]  David J. Kriegman,et al.  The Bas-Relief Ambiguity , 2004, International Journal of Computer Vision.

[18]  Jitendra Malik,et al.  Color Constancy, Intrinsic Images, and Shape Estimation , 2012, ECCV.

[19]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Stephen Lin,et al.  Estimation of Intrinsic Image Sequences from Image+Depth Video , 2012, ECCV.

[21]  Harry Shum,et al.  Image completion with structure propagation , 2005, ACM Trans. Graph..

[22]  Rui Huang,et al.  Shape-from-shading under complex natural illumination , 2011, 2011 18th IEEE International Conference on Image Processing.