CRFs for Image Classification

We use Conditional Random Fields (CRFs) to classify regions in an image. CRFs provide a discriminative framework to incorporate spatial dependencies in an image, which is more appropriate for classification tasks as opposed to a generative framework. In this paper we apply CRFs to two image classification tasks: a binary classification problem (manmade vs. natural regions in the Corel dataset), and a multiclass problem (grass, sky, tree, cow and building in the Microsoft Research, Cambridge dataset). Parameter learning is performed using Mean Field (MF) and Loopy Belief Propagation (LBP) to maximize an approximation to the conditional likelihood, and inference is done using LBP. We focus on three aspects of the classification task: feature extraction, feature aggregation, and techniques to combine binary classifiers to obtain multiclass classification. We present classification results on sample images from both datasets and provide analysis of the effects of various design choices on classification performance.