Detecting Specular Surfaces on Natural Images

Recognizing and localizing specular (or mirror-like) surfaces from a single image is a great challenge to computer vision. Unlike other materials, the appearance of a specular surface changes as function of the surrounding environment as well as the position of the observer. Even though the reflection on a specular surface has an intrinsic ambiguity that might be resolved by high level reasoning, we argue that we can take advantage of low level features to recognize specular surfaces. This intuition stems from the observation that the surrounding scene is highly distorted when reflected off regions of high curvature or occluding contours. We call these features static specular flows (SSF). We show how to characterize SSF and use them for identifying specular surfaces. To evaluate our result we collect a dataset of 120 images containing specular surfaces. Our algorithm can achieve good performances on this challenging dataset. Particularly, our results outperform other methods that follow a more naive approach.

[1]  A. Torralba,et al.  Specular reflections and the perception of shape. , 2004, Journal of vision.

[2]  Kristin J. Dana,et al.  Compact representation of bidirectional texture functions , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[3]  Jitendra Malik,et al.  Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Jitendra Malik,et al.  Normalized Cuts and Image Segmentation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  David W. Jacobs,et al.  Using specularities for recognition , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[7]  Michael J. Black,et al.  Specular Flow and the Recovery of Surface Structure , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[8]  Andrew Zisserman,et al.  Classifying Images of Materials: Achieving Viewpoint and Illumination Independence , 2002, ECCV.

[9]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[10]  J. Ross Quinlan,et al.  Bagging, Boosting, and C4.5 , 1996, AAAI/IAAI, Vol. 1.

[11]  Min Chen,et al.  Local Shape from Mirror Reflections , 2005, International Journal of Computer Vision.

[12]  E. Adelson,et al.  Recognition of Surface Reflectance Properties from a Single Image under Unknown Real-World Illumination , 2001 .

[13]  Pietro Perona,et al.  A Bayesian hierarchical model for learning natural scene categories , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[14]  Jitendra Malik,et al.  Representing and Recognizing the Visual Appearance of Materials using Three-dimensional Textons , 2001, International Journal of Computer Vision.

[15]  Heiko Neumann,et al.  Extracting and depicting the 3D shape of specular surfaces , 2005, APGV '05.

[16]  Silvio Savarese,et al.  What do reflections tell us about the shape of a mirror? , 2004, APGV '04.

[17]  David A. Forsyth,et al.  Finding glass , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[18]  Richard Szeliski,et al.  On the Motion and Appearance of Specularities in Image Sequences , 2002, ECCV.