A Language Modeling Approach to Image Classification

Due to the recent and fast diffusion of new digital devices (digital cameras, camera cell phones, internet), the number and size of image databases is dramatically increasing. Managing such databases is an important issue, for professional databases (e.g. from photo agencies) as well as for personal collections. Image classification and retrieval are therefore becoming more and more challenging. Discriminant image descriptors and robust classifiers are needed to handle these tasks. Nowadays approaches generally rely on describing images as a set of elementary and independent image patches called visual words, then using a classical classifier such as Support Vector Machines. In this paper, we propose a more precise description of images, called visual sentences, that includes simple spatial information between visual words. We then propose a classification technique based on language modeling. This classifier can exploit the spatial information of the visual sentences. Experiments on two classical datasets show that our classification method clearly outperforms the state-of- the-art SVM classifier.