Color Subspaces as Photometric Invariants

Abstract Complex reflectance phenomena such as specular reflections confound many vision problems since they produce image ‘features’ that do not correspond directly to intrinsic surface properties such as shape and spectral reflectance. A common approach to mitigate these effects is to explore functions of an image that are invariant to these photometric events. In this paper we describe a class of such invariants that result from exploiting color information in images of dichromatic surfaces. These invariants are derived from illuminant-dependent ‘subspaces’ of RGB color space, and they enable the application of Lambertian-based vision techniques to a broad class of specular, non-Lambertian scenes. Using implementations of recent algorithms taken from the literature, we demonstrate the practical utility of these invariants for a wide variety of applications, including stereo, shape from shading, photometric stereo, material-based segmentation, and motion estimation.

[1]  K. Ikeuchi,et al.  Color constancy through inverse-intensity chromaticity space. , 2004, Journal of the Optical Society of America. A, Optics, image science, and vision.

[2]  Ying-li Tian,et al.  Shape recovery from a color image for non-Lambertian surfaces , 1997 .

[3]  Takeo Kanade,et al.  The measurement of highlights in color images , 1988, International Journal of Computer Vision.

[4]  Glenn Healey,et al.  Using color for geometry-insensitive segmentation , 1989 .

[5]  Takeo Kanade,et al.  Determining shape and reflectance of hybrid surfaces by photometric sampling , 1989, IEEE Trans. Robotics Autom..

[6]  Katsushi Ikeuchi,et al.  Separating Reflection Components of Textured Surfaces Using a Single Image , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Brian V. Funt,et al.  A comparison of computational color constancy Algorithms. II. Experiments with image data , 2002, IEEE Trans. Image Process..

[8]  Brian V. Funt,et al.  A comparison of computational color constancy algorithms. I: Methodology and experiments with synthesized data , 2002, IEEE Trans. Image Process..

[9]  Terrance E. Boult,et al.  Constraining Object Features Using a Polarization Reflectance Model , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Berthold K. P. Horn,et al.  Determining Shape and Reflectance Using Multiple Images , 1978 .

[11]  Katsushi Ikeuchi,et al.  Temporal-color space analysis of reflection , 1994 .

[12]  Karsten Schlüns,et al.  Photometric Stereo for Non-Lambertian Surfaces Using Color Information , 1993, CAIP.

[13]  Reinhard Klette,et al.  Quantitative color optical flow , 2002, Object recognition supported by user interaction for service robots.

[14]  Stephen Lin,et al.  Diffuse-Specular Separation and Depth Recovery from Image Sequences , 2002, ECCV.

[15]  David J. Kriegman,et al.  Binocular Helmholtz stereopsis , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[16]  Shree K. Nayar,et al.  Reflectance based object recognition , 1996, International Journal of Computer Vision.

[17]  Shree K. Nayar,et al.  A class of photometric invariants: separating material from shape and illumination , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[18]  Jae Byung Park.  Efficient color representation for image segmentation under nonwhite illumination , 2003, SPIE Optics East.

[19]  Yee-Hong Yang,et al.  Shape from shading for non-Lambertian surfaces , 1994, Proceedings of 1st International Conference on Image Processing.

[20]  Graham D. Finlayson,et al.  Color by Correlation: A Simple, Unifying Framework for Color Constancy , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  T M Lehmann,et al.  Color line search for illuminant estimation in real-world scenes. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[22]  H C Lee,et al.  Method for computing the scene-illuminant chromaticity from specular highlights. , 1986, Journal of the Optical Society of America. A, Optics and image science.

[23]  Robert J. Woodham,et al.  Photometric Stereo: A Reflectance Map Technique For Determining Surface Orientation From Image Intensity , 1979, Optics & Photonics.

[24]  Karsten Schlfins.  Photometric Stereo for Non-Lambertian Surfaces Using Color Information , 2005 .

[25]  Carlo Tomasi,et al.  Depth Discontinuities by Pixel-to-Pixel Stereo , 1999, International Journal of Computer Vision.

[26]  David J. Kriegman,et al.  Image-based modeling and rendering of surfaces with arbitrary BRDFs , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[27]  Stephen Lin,et al.  Multibaseline stereo in the presence of specular reflections , 2002, Object recognition supported by user interaction for service robots.

[28]  Carlo Tomasi,et al.  Depth Discontinuities by Pixel-to-Pixel Stereo , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[29]  Steven M. Seitz,et al.  Shape and materials by example: a photometric stereo approach , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[30]  B. Wandell,et al.  Standard surface-reflectance model and illuminant estimation , 1989 .

[31]  Ruigang Yang,et al.  BRDF Invariant Stereo Using Light Transport Constancy , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Stefano Soatto,et al.  Multi-View Stereo Reconstruction of Dense Shape and Complex Appearance , 2005, International Journal of Computer Vision.

[33]  B. Wandell,et al.  Natural scene-illuminant estimation using the sensor correlation , 2002, Proc. IEEE.

[34]  Steven A. Shafer,et al.  Using color to separate reflection components , 1985 .

[35]  Michael J. Black,et al.  A framework for the robust estimation of optical flow , 1993, 1993 (4th) International Conference on Computer Vision.

[36]  Shree K. Nayar,et al.  Stereo and Specular Reflection , 1998, International Journal of Computer Vision.

[37]  Shree K. Nayar,et al.  Separation of Reflection Components Using Color and Polarization , 1997, International Journal of Computer Vision.

[38]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Norimichi Tsumura,et al.  Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin , 2003, ACM Trans. Graph..

[40]  Stephen Lin,et al.  Separation of Highlight Reflections on Textured Surfaces , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[41]  Maria Petrou,et al.  The 4-Source Photometric Stereo Technique for Three-Dimensional Surfaces in the Presence of Highlights and Shadows , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Rui J. P. de Figueiredo,et al.  A Theory of Photometric Stereo for a Class of Diffuse Non-Lambertian Surfaces , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  Andrew Blake,et al.  Detecting Specular Reflections Using Lambertian Constraints , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[44]  Gerald Schaefer,et al.  Constrained Dichromatic Colour Constancy , 2000, ECCV.

[45]  Steve Marschner,et al.  A practical model for subsurface light transport , 2001, SIGGRAPH.

[46]  Guillermo Sapiro,et al.  Color and Illuminant Voting , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[47]  Vladimir Kolmogorov,et al.  Visual correspondence using energy minimization and mutual information , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[48]  H. Johnson,et al.  A comparison of 'traditional' and multimedia information systems development practices , 2003, Inf. Softw. Technol..

[49]  In-So Kweon,et al.  Adaptive Support-Weight Approach for Correspondence Search , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[50]  Mark Galer.  High Dynamic Range , 2007 .

[51]  Li Zhang,et al.  Shape and motion under varying illumination: unifying structure from motion, photometric stereo, and multiview stereo , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[52]  David J. Kriegman,et al.  Specularity Removal in Images and Videos: A PDE Approach , 2006, ECCV.

[53]  R. Gershon.  The use of color in computational vision , 1987 .

[54]  Ruigang Yang,et al.  Dealing with textureless regions and specular highlights - a progressive space carving scheme using a novel photo-consistency measure , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[55]  David J. Kriegman,et al.  Beyond Lambert: reconstructing surfaces with arbitrary BRDFs , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[56]  Daniel Cremers,et al.  Shedding light on stereoscopic segmentation , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[57]  Peter H. Tu,et al.  Surface reconstruction via Helmholtz reciprocity with a single image pair , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[58]  E. North Coleman,et al.  Obtaining 3-dimensional shape of textured and specular surfaces using four-source photometry , 1982, Comput. Graph. Image Process..

[59]  David J. Kriegman,et al.  Helmholtz Stereopsis: Exploiting Reciprocity for Surface Reconstruction , 2002, International Journal of Computer Vision.

[60]  Mark S. Drew,et al.  Removing Shadows from Images , 2002, ECCV.

[61]  Maria Petrou,et al.  Colour photometric stereo: simultaneous reconstruction of local gradient and colour of rough textured surfaces , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[62]  Martial Hebert,et al.  Color Constancy Using KL-Divergence , 2001, ICCV.

[63]  Tan,et al.  Separating reflection components of textured surfaces using a single image , 2003, ICCV 2003.

[64]  James J. Little,et al.  Reflectance and Shape from Images Using a Collinear Light Source , 1999, International Journal of Computer Vision.

[65]  David J. Kriegman,et al.  Beyond Lambert: reconstructing specular surfaces using color , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[66]  Olga Veksler,et al.  Markov random fields with efficient approximations , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[67]  D H Brainard,et al.  Bayesian color constancy. , 1997, Journal of the Optical Society of America. A, Optics, image science, and vision.

[68]  In-So Kweon,et al.  Correspondence Search in the Presence of Specular Highlights Using Specular-Free Two-Band Images , 2006, ACCV.

[69]  Aly A. Farag,et al.  A New Formulation for Shape from Shading for Non-Lambertian Surfaces , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[70]  S. D. Hordley,et al.  Reevaluation of color constancy algorithm performance. , 2006, Journal of the Optical Society of America. A, Optics, image science, and vision.

[71]  Joost van de Weijer,et al.  Robust optical flow from photometric invariants , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[72]  Rama Chellappa,et al.  Estimation of illuminant direction, albedo, and shape from shading , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[73]  Graham D. Finlayson,et al.  Color in Perspective , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[74]  Edwin R. Hancock,et al.  Separating Lambertian and Specular Reflectance Components using Iterated Conditional Modes , 2001, BMVC.

[75]  Hsien-Che Lee,et al.  Modeling Light Reflection for Computer Color Vision , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[76]  Shree K. Nayar,et al.  High Dynamic Range from Multiple Images: Which Exposures to Combine?∗ , 2003 .

[77]  L. B. Wolff,et al.  Three-dimensional stereo by photometric ratios , 1994 .

[78]  Katsushi Ikeuchi,et al.  Determining Surface Orientations of Specular Surfaces by Using the Photometric Stereo Method , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[79]  David J. Kriegman,et al.  Passive photometric stereo from motion , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.