Birds of a feather flock together: Visual representation with scale and class consistency

Abstract There are three problems with a local-feature based representation scheme. First, local regions are often densely extracted or determined through detection without considering the scales of local regions. Second, local features are encoded separately, leaving the relationship among them unconsidered. Third, local features are simply encoded without considering the class information. To solve these problems, in this paper, we propose a scale and class consistent local-feature encoding method for image representation, which is achieved through the dense extraction of local features in different scale spaces, and the subsequent learning of the encoding parameters. In addition, instead of encoding each local feature independently, we jointly optimize the encoding parameters of the local features. Moreover, we also impose class consistency during the local-feature encoding process. We test the discriminative power of image representations on image classification tasks. Experiments on several public image datasets demonstrate that the proposed method achieves a superior performance compared with many other local-feature based methods.