A semantic model for video based face recognition

Video-based face recognition has attracted a great deal of attention in recent years due to its wide applications. The challenge of video-based face recognition comes from several aspects. First, video data involves many frames, which increases data size and processing complexity. Second, key frames extracted from videos are usually of high intra-personal discrepancy due to variations in expressions, poses, and illuminations. In order to address these problems, we propose a novel semantic based subspace model to improve the performance of video based face recognition. The basic idea is to construct an appropriate low-dimensional subspace for each person, upon which a semantic model is built to classify the key frames of the person into specific class. After the semantic classification, the key frames belonging to the same classes, i.e. the same semantics, are used to train the linear classifiers for recognition. Extensive experiments on a large face video database (XM2VTS) clearly show that our approach obtains a significant performance improvement over the traditional approaches.