Object Recognition: Theories

The goal of object recognition is to determine the identity or category of an object in a visual scene from the retinal input. In naturalistic scenes, object recognition is a computational challenge because the object may appear in various poses and contexts—i.e., in arbitrary positions, orientations, and distances with respect to the viewer and to other objects. Object recognition involves matching representations of objects stored in memory to representations extracted from the visual image. The key issue in object recognition is the nature of the representation extracted from the image. Theories of object recognition are characterized in terms of five logically independent dimensions: the primitive features or parts extracted from the visual image, the stability of the set of features to transformations of the image, the type of relationships used to describe configurations of features, and the stability of configurations across transformations of the image. The two main classes of object recognition theories—structural description and view-based theories—are characterized with respect to these dimensions, and examples of theories of each class are presented. The strengths and weaknesses of each class is discussed, and the argument is made that the two classes are in fact complementary, not antagonistic.