Color Subspaces as Photometric Invariants

Complex reflectance phenomena such as specular reflections confound many vision problems since they produce image ‘features’ that do not correspond directly to intrinsic surface properties such as shape and spectral reflectance. A common approach to mitigate these effects is to explore functions of an image that are invariant to these photometric events. In this paper we describe two such invariants" one invariant to specular reflections, and the other invariant to both specular reflections and diffuse shading" that result from exploiting color information in images of dichromatic surfaces. These invariants are derived from subspaces of RGB color space, and they enable the application of Lambertian-based vision techniques to a broad class of specular, non-Lambertian scenes. Using implementations of recent algorithms taken from the literature, we demonstrate the practical utility of these invariants for a wide variety of applications, including stereo, shape from shading, material-based segmentation, and motion estimation.

[1]  Berthold K. P. Horn,et al.  Determining Shape and Reflectance Using Multiple Images , 1978 .

[2]  Robert J. Woodham,et al.  Photometric Stereo: A Reflectance Map Technique For Determining Surface Orientation From Image Intensity , 1979, Optics & Photonics.

[3]  Katsushi Ikeuchi,et al.  Determining Surface Orientations of Specular Surfaces by Using the Photometric Stereo Method , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  E. North Coleman,et al.  Obtaining 3-dimensional shape of textured and specular surfaces using four-source photometry , 1982, Comput. Graph. Image Process..

[5]  Steven A. Shafer,et al.  Using color to separate reflection components , 1985 .

[6]  H C Lee,et al.  Method for computing the scene-illuminant chromaticity from specular highlights. , 1986, Journal of the Optical Society of America. A, Optics and image science.

[7]  R. Gershon.  The use of color in computational vision , 1987 .

[8]  Andrew Blake,et al.  Detecting Specular Reflections Using Lambertian Constraints , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[9]  Glenn Healey,et al.  Using color for geometry-insensitive segmentation , 1989 .

[10]  B. Wandell,et al.  Standard surface-reflectance model and illuminant estimation , 1989 .

[11]  Hsien-Che Lee,et al.  Modeling Light Reflection for Computer Color Vision , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Takeo Kanade,et al.  Determining shape and reflectance of hybrid surfaces by photometric sampling , 1989, IEEE Trans. Robotics Autom..

[13]  Terrance E. Boult,et al.  Constraining Object Features Using a Polarization Reflectance Model , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Rama Chellappa,et al.  Estimation of Illuminant Direction, Albedo, and Shape from Shading , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Rui J. P. de Figueiredo,et al.  A Theory of Photometric Stereo for a Class of Diffuse Non-Lambertian Surfaces , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Karsten Schlüns,et al.  Photometric Stereo for Non-Lambertian Surfaces Using Color Information , 1993, CAIP.

[17]  Michael J. Black,et al.  A framework for the robust estimation of optical flow , 1993, 1993 (4th) International Conference on Computer Vision.

[18]  L. B. Wolff,et al.  Three-dimensional stereo by photometric ratios , 1994 .

[19]  Yee-Hong Yang,et al.  Shape from shading for non-Lambertian surfaces , 1994, Proceedings of 1st International Conference on Image Processing.

[20]  Katsushi Ikeuchi,et al.  Temporal-color space analysis of reflection , 1994 .

[21]  Graham D. Finlayson,et al.  Color in Perspective , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Ying-li Tian,et al.  Shape recovery from a color image for non-Lambertian surfaces , 1997 .

[23]  D H Brainard,et al.  Bayesian color constancy. , 1997, Journal of the Optical Society of America. A, Optics, image science, and vision.

[24]  Carlo Tomasi,et al.  Depth Discontinuities by Pixel-to-Pixel Stereo , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[25]  Olga Veksler,et al.  Markov random fields with efficient approximations , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[26]  Guillermo Sapiro,et al.  Color and Illuminant Voting , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Gerald Schaefer,et al.  Constrained Dichromatic Colour Constancy , 2000, ECCV.

[28]  Graham D. Finlayson,et al.  Color by Correlation: A Simple, Unifying Framework for Color Constancy , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  David J. Kriegman,et al.  Image-based modeling and rendering of surfaces with arbitrary BRDFs , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[30]  T M Lehmann,et al.  Color line search for illuminant estimation in real-world scenes. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[31]  Edwin R. Hancock,et al.  Separating Lambertian and Specular Reflectance Components using Iterated Conditional Modes , 2001, BMVC.

[32]  Steve Marschner,et al.  A practical model for subsurface light transport , 2001, SIGGRAPH.

[33]  Color constancy using KL-divergence , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[34]  David J. Kriegman,et al.  Beyond Lambert: reconstructing surfaces with arbitrary BRDFs , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[35]  Maria Petrou,et al.  Colour photometric stereo: simultaneous reconstruction of local gradient and colour of rough textured surfaces , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[36]  Mark S. Drew,et al.  Removing Shadows from Images , 2002, ECCV.

[37]  Stephen Lin,et al.  Diffuse-Specular Separation and Depth Recovery from Image Sequences , 2002, ECCV.

[38]  David J. Kriegman,et al.  Helmholtz Stereopsis: Exploiting Reciprocity for Surface Reconstruction , 2002, International Journal of Computer Vision.

[39]  Brian V. Funt,et al.  A comparison of computational color constancy algorithms. I: Methodology and experiments with synthesized data , 2002, IEEE Trans. Image Process..

[40]  B. Wandell,et al.  Natural scene-illuminant estimation using the sensor correlation , 2002, Proc. IEEE.

[41]  Reinhard Klette,et al.  Quantitative color optical flow , 2002, Object recognition supported by user interaction for service robots.

[42]  Brian V. Funt,et al.  A comparison of computational color constancy Algorithms. II. Experiments with image data , 2002, IEEE Trans. Image Process..

[43]  Stephen Lin,et al.  Multibaseline stereo in the presence of specular reflections , 2002, Object recognition supported by user interaction for service robots.

[44]  Vladimir Kolmogorov,et al.  Visual correspondence using energy minimization and mutual information , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[45]  Ruigang Yang,et al.  Dealing with textureless regions and specular highlights - a progressive space carving scheme using a novel photo-consistency measure , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[46]  Shree K. Nayar,et al.  A class of photometric invariants: separating material from shape and illumination , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[47]  Shree K. Nayar,et al.  High Dynamic Range from Multiple Images: Which Exposures to Combine?∗ , 2003 .

[48]  Maria Petrou,et al.  The 4-Source Photometric Stereo Technique for Three-Dimensional Surfaces in the Presence of Highlights and Shadows , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  Li Zhang,et al.  Shape and motion under varying illumination: unifying structure from motion, photometric stereo, and multiview stereo , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[50]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  Steven M. Seitz,et al.  Shape and materials by example: a photometric stereo approach , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[52]  David J. Kriegman,et al.  Binocular Helmholtz stereopsis , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[53]  Peter H. Tu,et al.  Surface reconstruction via Helmholtz reciprocity with a single image pair , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[54]  Norimichi Tsumura,et al.  Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin , 2003, ACM Trans. Graph..

[55]  Jae Byung Park.  Efficient color representation for image segmentation under nonwhite illumination , 2003, SPIE Optics East.

[56]  Tan,et al.  Separating reflection components of textured surfaces using a single image , 2003, ICCV 2003.

[57]  Stefano Soatto,et al.  Shedding light on stereoscopic segmentation , 2004, CVPR 2004.

[58]  Shree K. Nayar,et al.  Reflectance based object recognition , 1996, International Journal of Computer Vision.

[59]  Shree K. Nayar,et al.  Stereo and Specular Reflection , 1998, International Journal of Computer Vision.

[60]  Shree K. Nayar,et al.  Separation of Reflection Components Using Color and Polarization , 1997, International Journal of Computer Vision.

[61]  James J. Little,et al.  Reflectance and Shape from Images Using a Collinear Light Source , 1999, International Journal of Computer Vision.

[62]  Robby T. Tan,et al.  Color constancy through inverse-intensity chromaticity space. , 2004 .

[63]  Joost van de Weijer,et al.  Robust optical flow from photometric invariants , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[64]  Takeo Kanade,et al.  The measurement of highlights in color images , 1988, International Journal of Computer Vision.

[65]  David J. Kriegman,et al.  Beyond Lambert: reconstructing specular surfaces using color , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[66]  David J. Kriegman,et al.  Passive photometric stereo from motion , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[67]  Stefano Soatto,et al.  Multi-View Stereo Reconstruction of Dense Shape and Complex Appearance , 2005, International Journal of Computer Vision.

[68]  Katsushi Ikeuchi,et al.  Separating Reflection Components of Textured Surfaces Using a Single Image , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[69]  In-So Kweon,et al.  Adaptive Support-Weight Approach for Correspondence Search , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[70]  Stephen Lin,et al.  Separation of Highlight Reflections on Textured Surfaces , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[71]  In-So Kweon,et al.  Correspondence Search in the Presence of Specular Highlights Using Specular-Free Two-Band Images , 2006, ACCV.

[72]  Aly A. Farag,et al.  A New Formulation for Shape from Shading for Non-Lambertian Surfaces , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[73]  S. D. Hordley,et al.  Reevaluation of color constancy algorithm performance. , 2006, Journal of the Optical Society of America. A, Optics, image science, and vision.

[74]  David J. Kriegman,et al.  Specularity Removal in Images and Videos: A PDE Approach , 2006, ECCV.

[75]  Ruigang Yang,et al.  BRDF Invariant Stereo Using Light Transport Constancy , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.