Embedding metric learning into set-based face recognition for video surveillance

Abstract Face recognition in video surveillance is a challenging task, largely due to the difficulty in matching images across cameras of distinct viewpoints and illuminations. To overcome this difficulty, this paper proposes a novel method which embeds distance metric learning into set-based image matching. First we use sets of face images, rather than individual images, as the input for recognition, since in surveillance systems the former is a more natural way. We model each image set using a convex-hull space spanned by its member images and measure the dissimilarity of two sets using the distance between the closest points of their corresponding convex hull spaces. Then we propose a set-based distance metric learning scheme to learn a feature-space mapping to a discriminative subspace. Finally we project image sets into the learned subspace and achieve face recognition by comparing the projected sets. In this way, we can adapt the variation in viewpoints and illuminations across cameras in order to improve face recognition in video surveillance. Experiments on the public Honda/UCSD and ChokePoint databases demonstrate the superior performance of our method to the state-of-the-art approaches.