The quotient image: Class based recognition and synthesis under varying illumination conditions

The paper addresses the problem of "class-based" recognition and image-synthesis with varying illumination. The class-based synthesis and recognition tasks are defined as follows: given a single input image of an object, and a sample of images with varying illumination conditions of other objects of the same general class, capture the equivalence relationship (by generation of new images or by invariants) among all images of the object corresponding to new illumination conditions. The key result in our approach is based on a definition of an illumination invariant signature image, we call the "quotient" image, which enables an analytic generation of the image space with varying illumination from a single input image and a very small sample of other objects of the class-in our experiments as few as two objects. In many cases the recognition results outperform by far conventional methods and the image-synthesis is of remarkable quality considering the size of the database of example images and the mild pre-process required for making the algorithm work.

[1]  C. M. Mooney.  Recognition of ambiguous and unambiguous visual configurations with short and longer exposures. , 1960, British journal of psychology.

[2]  Berthold K. P. Horn.  Image Intensity Understanding , 1975 .

[3]  Orthogonal Transform for Digital Signal Processing , 1976, IEEE Transactions on Systems, Man, and Cybernetics.

[4]  Wayne D. Gray,et al.  Basic objects in natural categories , 1976, Cognitive Psychology.

[5]  A R Smith,et al.  Color Gamut Transformation Pairs , 1978 .

[6]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[7]  Katsushi Ikeuchi,et al.  Numerical Shape from Shading and Occluding Boundaries , 1981, Artif. Intell..

[8]  A. Pentland.  Finding the illuminant direction , 1982 .

[9]  Alex Pentland,et al.  Local Shading Analysis , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Michael J. Brooks,et al.  The variational approach to shape from shading , 1986, Comput. Vis. Graph. Image Process..

[11]  L Sirovich,et al.  Low-dimensional procedure for the characterization of human faces. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[12]  M. Farah.  Is visual imagery really visual? Overlooked evidence from neuropsychology. , 1988, Psychological review.

[13]  Y. J. Tejwani,et al.  Robot vision , 1989, IEEE International Symposium on Circuits and Systems,.

[14]  P. Fiildihk,et al.  Forming sparse representations by local anti-Hebbian learning , 1990 .

[15]  Amnon Shashua,et al.  Illumination and View Position in 3D Visual Recognition , 1991, NIPS.

[16]  Takeo Kanade,et al.  Surface Reflection: Physical and Geometrical Perspectives , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[18]  Ronen Basri,et al.  Recognition by Linear Combinations of Models , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  T. Poggio,et al.  Recognition and Structure from one 2D Model View: Observations on Prototypes, Object Classes and Symmetries , 1992 .

[20]  Erkki Oja,et al.  Principal components, minor components, and linear neural networks , 1992, Neural Networks.

[21]  C Tomasi,et al.  Shape and motion from image streams: a factorization method. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[22]  Ronen Basri,et al.  Recognition by prototypes , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Vishvjit S. Nalwa,et al.  A guided tour of computer vision , 1993 .

[24]  L. B. Wolff,et al.  3-D Stereo Using Photometric Ratios , 1994, ECCV.

[25]  Peter W. Hallinan.  A low-dimensional representation of human faces for arbitrary lighting conditions , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Julie Dorsey,et al.  Effic ient Re-rendering of Naturally Illuminated Environments , 1994 .

[27]  David J. Field,et al.  What Is the Goal of Sensory Coding? , 1994, Neural Computation.

[28]  Vladimir Vapnik,et al.  The Nature of Statistical Learning , 1995 .

[29]  David J. Field,et al.  Sparse Coding of Natural Images Produces Localized, Oriented, Bandpass Receptive Fields , 1995 .

[30]  Rama Chellappa,et al.  Human and machine recognition of faces: a survey , 1995, Proc. IEEE.

[31]  Duncan Rowland,et al.  Manipulating facial appearance through shape and color , 1995, IEEE Computer Graphics and Applications.

[32]  S. Ullman,et al.  Generalization to Novel Images in Upright and Inverted Faces , 1993, Perception.

[33]  Tomaso A. Poggio,et al.  Image Synthesis from a Single Example Image , 1996, ECCV.

[34]  Tomaso Poggio,et al.  Image Representations for Visual Learning , 1996, Science.

[35]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[36]  Joshua B. Tenenbaum,et al.  Separating Style and Content , 1996, NIPS.

[37]  Joshua B. Tenenbaum,et al.  Learning bilinear models for two-factor problems in vision , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[38]  Shimon Ullman,et al.  Face Recognition: The Problem of Compensating for Changes in Illumination Direction , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Daniel Snow,et al.  Shape and albedo from multiple images using integrability , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[40]  Lawrence B. Wolff,et al.  Surface Curvature and Shape Reconstruction from Unknown Multiple Illumination and Integrability , 1997, Comput. Vis. Image Underst..

[41]  Tomaso A. Poggio,et al.  Linear Object Classes and Image Synthesis From a Single Example Image , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[43]  Tomaso A. Poggio,et al.  A bootstrapping algorithm for learning linear models of object classes , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[44]  Shimon Ullman,et al.  Recognizing novel 3-D objects under new illumination and viewing position using a small number of example views or even a single view , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[45]  Thomas Vetter,et al.  Estimating Coloured 3D Face Models from Single Images: An Example Based Approach , 1998, ECCV.

[46]  David J. Kriegman,et al.  Illumination cones for recognition under variable lighting: faces , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[47]  Erez Sali,et al.  Recognizing Novel 3-D Objects Under New Illumination and Viewing Position Using a Small Number of Examples. , 1998, ICCV 1998.

[48]  Jean-Pierre Nadal,et al.  Sparsely coded neural networks , 1998 .

[49]  M. Tarr,et al.  Do viewpoint-dependent mechanisms generalize across members of a class? , 1998, Cognition.

[50]  Peter Meer,et al.  Estimation with bilinear constraints in computer vision , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[51]  Amnon Shashua,et al.  The Quotient Image: Class-Based Re-Rendering and Recognition with Varying Illuminations , 2001, IEEE Trans. Pattern Anal. Mach. Intell..