Object Recognition with Latent Conditional Random Fields

In this thesis we present a discriminative part-based approach for the recognition of object classes from unsegmented cluttered scenes. Objects are modelled as flexible constellations of parts conditioned on local observations. For each object class the probability of a given assignment of parts to local features is modelled by a Conditional Random Field (CRF). We propose an extension of the CRF framework that incorporates hidden variables and combines class conditional CRFs into a unified framework for part-based object recognition. The random field captures spatial coherence between region labels. The parameters of the CRF are estimated in a maximum likelihood framework and recognition proceeds by finding the most likely class under our model. The main advantage of the proposed CRF framework is that it allows us to relax the assumption of conditional independence of the observed data (i.e. local features) often used in generative approaches, an assumption that might be too restrictive for a considerable number of object classes. In the second part of this work we extend the detection model and develop a discriminative recognition system which both detects the presence of objects and finds their regions of support in an image. Our part based model allows joint object detection and region labelling; in contrast to previous methods ours can be trained with a combination of examples for which we have labelled support regions and examples for which we only know whether the object is present in the image. We extend the detection model by incorporating a segmentation variable; the segmentation variable is assumed to be observed in the fully labelled data and hidden on the partially labelled one. Our latent variable model learns sets of part labels for each image site, which allows us to merge part-based detection with part-based region labelling (or segmentation). Thesis Supervisor: Michael Collins Title: Associate Professor Thesis Supervisor: Trevor Darrell Title: Associate Professor