Joint appearance and locality image representation by gaussianization

A novel image representation is proposed in this thesis to capture both the appearance and locality information for image classification applications. First, we model the feature vectors, from various granularity levels including the corpus level, the image level and image patch level, in a hierarchical Bayesian framework using mixtures of Gaussians. After such a hierarchical Gaussianization, each image is represented as a Gaussian mixture model (GMM) for its appearance, and several Gaussian maps for its spatial layout. Then we extract the appearance information from the GMMparameters, and the locality information from the global and the local statistics over Gaussian maps. Finally, we employ a supervised dimension reduction technique called DAP (discriminant adaptive projection) to remove noise directions and to further enhance the discriminating power of our representation. To validate the argument that the new representation is a general representation for images and video frames, we evaluate the representation on several important applications. Firstly, we apply the new presentation to classification and regression tasks taking whole images as inputs. These tasks include object recognition, scene category classification, face recognition, age estimation, pose estimation, gender recognition, and video event recognition. Then we test it for the object detection and image parsing tasks, where the new representation takes partial images as inputs. The experimental results show that, for various types of images and tasks, the performances using the proposed representation were the best in all the applications compared with other state-of-the-art algorithms.