Indexing with unknown illumination and pose

The task of identifying 3D objects in 2D images is difficult due to variation in objects' appearance with changes in pose and lighting. The task is further complicated by the presence of occlusion and clutter. Shape indexing is a method for rapid association between features identified in an image and their corresponding 3D features stored in a database. Previous indexing methods ignored variations due to lighting, restricting the approach to polyhedral objects. In this paper, we further develop these methods to handle variations in both pose and lighting. We focus on rigid objects undergoing a scaled-orthographic projection and use spherical harmonics to represent lighting. The resulting integrated algorithm can recognize 3D objects from a single input image; furthermore, it recovers the pose and lighting of each familiar object in the given image. The algorithm has been tested on a database of real objects, demonstrating its performance on cluttered scenes under a variety of poses and illumination conditions.

[1]  Daphna Weinshall.  Model-based invariants for 3-D vision , 2005, International Journal of Computer Vision.

[2]  Ronen Basri,et al.  Lambertian Reflectance and Linear Subspaces , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[5]  David A. Forsyth,et al.  Invariant Descriptors for 3D Object Recognition and Pose , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  David J. Kriegman,et al.  From few to many: generative models for recognition under variable pose and illumination , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[7]  Jean Ponce,et al.  Probabilistic 3D Object Recognition , 2004, International Journal of Computer Vision.

[8]  David W. Jacobs.  Matching 3-D Models to 2-D Images , 2004, International Journal of Computer Vision.

[9]  David W. Jacobs,et al.  Space and Time Bounds on Indexing 3D Models from 2D Images , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[11]  J.B. Burns,et al.  View Variation of Point-Set and Line-Segment Features , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Andrew Zisserman,et al.  Geometric invariance in computer vision , 1992 .

[13]  David J. Kriegman,et al.  Invariant-based recognition of complex curved 3D objects from image contours , 1995, Proceedings of IEEE International Conference on Computer Vision.

[14]  Clark F. Olson.  Probabilistic Indexing for Object Recognition , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  P. Hanrahan,et al.  On the relationship between radiance and irradiance: determining the illumination from images of a convex Lambertian object. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[16]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Paul A. Viola,et al.  Alignment by Maximization of Mutual Information , 1997, International Journal of Computer Vision.

[18]  David G. Lowe,et al.  Indexing without Invariants in 3D Object Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Daphna Weinshall.  Model-based invariants for 3D vision , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Shimon Ullman,et al.  Limitations of Non Model-Based Recognition Schemes , 1992, ECCV.

[21]  Ralph Gross,et al.  Eigen light-fields and face recognition across pose , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[22]  Hiroshi Murase,et al.  Learning and recognition of 3D objects from appearance , 1993, [1993] Proceedings IEEE Workshop on Qualitative Vision.

[23]  Yehezkel Lamdan,et al.  Geometric Hashing: A General And Efficient Model-based Recognition Scheme , 1988, [1988 Proceedings] Second International Conference on Computer Vision.