Shape-based image segmentation through photometric stereo

This paper describes a new algorithm for segmenting 2D images by taking into account 3D shape information. The proposed approach consists of two stages. In the first stage, the 3D surface normals of the objects present in the scene are estimated through robust photometric stereo. Then, the image is segmented by grouping its pixels according to their estimated normals through graph-based clustering. One of the advantages of the proposed approach is that, although the segmentation is based on the 3D shape of the objects, the photometric stereo stage used to estimate the 3D normals only requires a set of 2D images. This paper provides an extensive validation of the proposed approach by comparing it with several image segmentation algorithms. Particularly, it is compared with both appearance-based image segmentation algorithms and shape-based ones. Experimental results confirm that the latter are more suitable when the objective is to segment the objects or surfaces present in the scene. Moreover, results show that the proposed approach yields the best image segmentation in most of the cases.

[1]  Andrew W. Fitzgibbon,et al.  An Experimental Comparison of Range Image Segmentation Algorithms , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[3]  Srinivasa G. Narasimhan,et al.  Clustering Appearance for Scene Analysis , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[4]  Jitendra Malik,et al.  From contours to regions: An empirical evaluation , 2009, CVPR.

[5]  Ira Kemelmacher-Shlizerman,et al.  Photometric Stereo with General, Unknown Lighting , 2006, International Journal of Computer Vision.

[6]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[7]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[8]  Joan Serrat,et al.  Photometric stereo through an adapted alternation approach , 2008, 2008 15th IEEE International Conference on Image Processing.

[9]  R. Hartley,et al.  PowerFactorization : 3D reconstruction with missing or uncertain data , 2003 .

[10]  Dima Damen,et al.  Recognizing linked events: Searching the space of feasible explanations , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Andrew W. Fitzgibbon,et al.  Damped Newton algorithms for matrix factorization with missing data , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[12]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Kosuke Sato,et al.  Some further results of experimental comparison of range image segmentation algorithms , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[14]  Daniel Snow,et al.  Determining Generative Models of Objects Under Varying Illumination: Shape and Albedo from Multiple Images Using SVD and Integrability , 1999, International Journal of Computer Vision.

[15]  M. García,et al.  Image segmentation through graph-based clustering from surface normals estimated by photometric stereo , 2010 .

[16]  B. S. Manjunath,et al.  A Variational Framework for Multiregion Pairwise-Similarity-Based Image Segmentation , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Hideki Hayakawa.  Photometric stereo under a light source with arbitrary motion , 1994 .

[18]  Miguel Ángel García,et al.  Fast extraction of surface primitives from range images , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[19]  Chi-Keung Tang,et al.  A Computational Framework for Feature Extraction and Segmentation , 2000 .

[20]  Yasuyuki Matsushita,et al.  Self-calibrating photometric stereo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Srinivasa G. Narasimhan,et al.  Appearance Derivatives for Isonormal Clustering of Scenes , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Oliver Wirjadi,et al.  Survey of 3d image segmentation methods , 2007 .

[23]  Pedro M. Q. Aguiar,et al.  Estimation of Rank Deficient Matrices from Partial Observations: Two-Step Iterative Algorithms , 2003, EMMCVPR.

[24]  Domenec Puig,et al.  Graph-Based Perceptual Segmentation of Stereo Vision 3D Images at Multiple Abstraction Levels , 2007, GbRPR.

[25]  Jitendra Malik,et al.  From contours to regions: An empirical evaluation , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Allen Y. Yang,et al.  Unsupervised segmentation of natural images via lossy data compression , 2008, Comput. Vis. Image Underst..