A class of photometric invariants: separating material from shape and illumination

We derive a new class of photometric invariants that can be used for a variety of vision tasks including lighting invariant material segmentation, change detection and tracking, as well as material invariant shape recognition. The key idea is the formulation of a scene radiance model for the class of "separable" BRDFs, that can be decomposed into material related terms and object shape and lighting related terms. All the proposed invariants are simple rational functions of the appearance parameters (say, material or shape and lighting). The invariants in this class differ from one another in the number and type of image measurements they require. Most of the invariants in this class need changes in illumination or object position between image acquisitions. The invariants can handle large changes in lighting which pose problems for most existing vision algorithms. We demonstrate the power of these invariants using scenes with complex shapes, materials, textures, shadows and specularities.

[1]  L. B. Wolff,et al.  Segmentation of surface curvature with a photometric invariant , 1994 .

[2]  Cordelia Schmid,et al.  Local Grayvalue Invariants for Image Retrieval , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Shree K. Nayar,et al.  Generalization of the Lambertian model and implications for machine vision , 1995, International Journal of Computer Vision.

[4]  Arnold W. M. Smeulders,et al.  Measurement of color invariants , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[5]  Edward H. Adelson,et al.  Separating reflections and lighting using independent components analysis , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[6]  Edward M. Riseman,et al.  The non-existence of general-case view-invariants , 1992 .

[7]  Steven A. Shafer,et al.  Using color to separate reflection components , 1985 .

[8]  Peter W. Hallinan.  A low-dimensional representation of human faces for arbitrary lighting conditions , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Neil A. Dodgson,et al.  Proceedings Ninth IEEE International Conference on Computer Vision , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[10]  Arnold W. M. Smeulders,et al.  Color constant ratio gradients for image segmentation and similarity of texture objects , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[11]  K. Ikeuchi,et al.  Reflectance analysis under solar illumination , 1994, Proceedings of the Workshop on Physics-Based Modeling in Computer Vision.

[12]  David W. Jacobs,et al.  In search of illumination invariants , 2001, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[13]  Mark S. Drew,et al.  Removing Shadows from Images , 2002, ECCV.

[14]  Yair Weiss,et al.  Deriving intrinsic images from image sequences , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[15]  Ramakant Nevatia,et al.  Three-Dimensional Descriptions Based on the Analysis of the Invariant and Quasi-Invariant Properties of Some Curved-Axis Generalized Cylinders , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  J. Koenderink,et al.  Photometric Invariants Related to Solid Shape , 1980 .

[17]  Sang Wook Lee,et al.  An appearance representation for multiple reflection components , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[18]  Katsushi Ikeuchi,et al.  Temporal-color space analysis of reflection , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[19]  K. Torrance,et al.  Theory for off-specular reflection from roughened surfaces , 1967 .

[20]  Yasuyuki Matsushita,et al.  Shadow elimination for robust video surveillance , 2002, Workshop on Motion and Video Computing, 2002. Proceedings..

[21]  Sang Wook Lee,et al.  Estimation of diffuse and specular appearance , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[22]  Shree K. Nayar,et al.  Reflectance based object recognition , 1996, International Journal of Computer Vision.

[23]  Takeo Kanade,et al.  Surface Reflection: Physical and Geometrical Perspectives , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Glenn Healey,et al.  The Illumination-Invariant Recognition of 3D Objects Using Local Color Invariants , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Stephen J. Maybank,et al.  Probabilistic analysis of the application of the cross ratio to model based vision , 1995, International Journal of Computer Vision.

[26]  David J. Kriegman,et al.  What is the set of images of an object under all possible lighting conditions? , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[27]  Gudrun Klinker,et al.  A physical approach to color image understanding , 1989, International Journal of Computer Vision.

[28]  Katsushi Ikeuchi,et al.  Object shape and reflectance modeling from observation , 1997, SIGGRAPH.

[29]  Robert M. Haralick,et al.  Propagating covariance in computer vision , 1994, Proceedings of 12th International Conference on Pattern Recognition.