Limitations of using bags of complex features: Hierarchical higher-order filters fail to capture spatial configurations

One common method of representing images is to reduce an image to a collection of features. Many simple features have been proposed, such as pixel intensities and wavelet responses, but these choices are fundamentally unsuitable for capturing the configural relations of objects and object parts, as spatial information associated with each feature is lost. Another recent strategy, known as “feature-hierarchy” modeling, involves the use of overlapping, redundant features. These features are obtained by processing an image across a hierarchy of units tuned to progressively more complex properties. An open question is whether such approaches produce data structures rich enough for implicitly capturing configural relations. We implemented three experiments and several computer simulations to address this issue. Our method involved the use of four classes of objects, each derived from the simple spatial relationships present in classic Vernier and bisection acuity tasks. All human observers achieved near perfect categorization performance after relatively few exposures to each stimulus class. This ability also transferred across several dimensions, including orientation and background context. By contrast, simulations on a featurehierarchy model revealed poor performance for this class of models. Furthermore, the moderate categorization accuracy achieved did not transfer across even the simplest of dimensions. These results indicate that this approach to image representation lacks a fundamental property necessary for encoding the spatial configurations of object parts.