Color Photometric Stereo Using Multi-Band Camera Constrained by Median Filter and Occluding Boundary

One of the main problems faced by the photometric stereo method is that several measurements are required, as this method needs illumination from light sources from different directions. A solution to this problem is the color photometric stereo method, which conducts one-shot measurements by simultaneously illuminating lights of different wavelengths. However, the classic color photometric stereo method only allows measurements of white objects, while a surface-normal estimation of a multicolored object using this method is theoretically impossible. Therefore, it is necessary to add some constraints to estimate the surface normal of a multicolored object using the framework of the color photometric stereo method. In this study, a median filter is employed as the constraint condition of albedo, and the surface normal of the occluding boundary is employed as the constraint condition of the surface normal. By employing a median filter as the constraint condition, the smooth distribution of the albedo and normal is calculated while the sharp features at the boundary of different albedos and normals are preserved. The surface normal at the occluding boundary is propagated into the inner part of the object region, and forms the abstract shape of the object. Such a surface normal gives a great clue to be used as an initial guess to the surface normal. To demonstrate the effectiveness of this study, a measurement device that can realize the multispectral photometric stereo method with seven colors is employed instead of the classic color photometric stereo method with three colors.

[1]  Graham Fyffe,et al.  Single-shot photometric stereo by spectral multiplexing , 2010, 2011 IEEE International Conference on Computational Photography (ICCP).

[2]  Jiuai Sun,et al.  Object surface recovery using a multi-light photometric stereo technique for non-Lambertian surfaces subject to shadows and specularities , 2007, Image Vis. Comput..

[3]  Kalyan Sunkavalli,et al.  Single-Image RGB Photometric Stereo with Spatially-Varying Albedo , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[4]  L. L..  Reconstruction of shape from shading in color images , .

[5]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[6]  Björn Stenger,et al.  Non-rigid Photometric Stereo with Colored Lights , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[7]  Jean-Denis Durou,et al.  Normal Integration: A Survey , 2017, Journal of Mathematical Imaging and Vision.

[8]  Yaser Sheikh,et al.  Photogeometric Scene Flow for High-Detail Dynamic 3D Reconstruction , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[9]  Björn Stenger,et al.  Color photometric stereo for multicolored surfaces , 2011, 2011 International Conference on Computer Vision.

[10]  Håkan Jonsson,et al.  Sub-Millimeter Crack Detection in Casted Steel Using Color Photometric Stereo , 2013, 2013 International Conference on Digital Image Computing: Techniques and Applications (DICTA).

[11]  Giovanni Maria Farinella,et al.  Computer Vision: Detection, Recognition and Reconstruction , 2010, Computer Vision: Detection, Recognition and Reconstruction.

[12]  Jean-Yves Guillemaut,et al.  Colour Helmholtz Stereopsis for Reconstruction of Complex Dynamic Scenes , 2014, 2014 2nd International Conference on 3D Vision.

[13]  Roberto Cipolla,et al.  Shadows in Three-Source Photometric Stereo , 2008, ECCV.

[14]  Jean-Denis Durou,et al.  Unbiased Photometric Stereo for Colored Surfaces: A Variational Approach , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Mark S. Drew,et al.  Closed-form attitude determination under spectrally varying illumination , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Carlos Hernández,et al.  Practical 3D Reconstruction Based on Photometric Stereo , 2010, Computer Vision: Detection, Recognition and Reconstruction.

[17]  Katsushi Ikeuchi,et al.  Numerical Shape from Shading and Occluding Boundaries , 1981, Artif. Intell..

[18]  Michael Breuß,et al.  Fast and accurate surface normal integration on non-rectangular domains , 2016, Computational Visual Media.

[19]  Xavier Descombes,et al.  Photometric stereo with only two images: A theoretical study and numerical resolution , 2017, Image Vis. Comput..

[20]  Björn Stenger,et al.  Video Normals from Colored Lights , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  George Vogiatzis,et al.  Self-calibrated, Multi-spectral Photometric Stereo for 3D Face Capture , 2012, International Journal of Computer Vision.

[22]  Rama Chellappa,et al.  A Method for Enforcing Integrability in Shape from Shading Algorithms , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Daniel Cremers,et al.  Fast and globally optimal single view reconstruction of curved objects , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Moshe Ben-Ezra,et al.  Photometric Stereo for Dynamic Surface Orientations , 2010, ECCV.

[25]  Imari Sato,et al.  Color Photometric Stereo Using a Rainbow Light for Non-Lambertian Multicolored Surfaces , 2014, ACCV.

[26]  Berthold K. P. Horn,et al.  Determining Shape and Reflectance Using Multiple Images , 1978 .

[27]  G D Finlayson,et al.  Spectral sharpening: sensor transformations for improved color constancy. , 1994, Journal of the Optical Society of America. A, Optics, image science, and vision.

[28]  Katsushi Ikeuchi,et al.  Color estimation from a single surface color , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  K Barnard,et al.  Sensor sharpening for computational color constancy. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[30]  Mark S. Drew.  Direct Solution of Orientation-from-Color Problem Using a Modification of Pentland's Light Source Direction Estimator , 1996, Comput. Vis. Image Underst..

[31]  R. Woodham.  Gradient and curvature from the photometric-stereo method, including local confidence estimation , 1994 .

[32]  Junyu Dong,et al.  Underwater multi-spectral photometric stereo reconstruction from a single RGBD image , 2016, 2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA).

[33]  R. Woodham.  Gradient and Curvature from Photometric Stereo Including Local Condence Estimation , 1994 .

[34]  A. P. Petrov,et al.  Properties of color images of surfaces under multiple illuminants , 1994 .

[35]  Drew,et al.  Color from shape from color: a simple formalism with known light sources , 2000, Journal of the Optical Society of America. A, Optics, image science, and vision.

[36]  Mark S. Drew.  Reduction of rank-reduced orientation-from-color problem with many unknown lights to two-image known-illuminant photometric stereo , 1995, Proceedings of International Symposium on Computer Vision - ISCV.

[37]  Michael J. Brooks,et al.  The variational approach to shape from shading , 1986, Comput. Vis. Graph. Image Process..