Multiple appearance models for face tracking in surveillance videos

Face tracking is a key component for automated video surveillance systems. It supports and enhances tasks such as face recognition and video indexing. Face tracking in surveillance scenarios is a challenging problem due to ambient illumination variations, face pose changes, occlusions, and background clutter. We present an algorithm for tracking faces in surveillance video based on a particle filter mechanism using multiple appearance models for robust representation of the face. We propose color based appearance model complemented by an edge based appearance model using the Difference of Gaussian (DOG) filters. We demonstrate that combined appearance models are more robust in handling the face and scene variations than a single appearance model. For example, color template appearance model is better in handling pose variations but they deteriorate against illumination variations. Similarly, an edge based model is robust in handling illumination variations but they fail in handling substantial pose changes. Hence, a combined model is more robust in handling pose and illumination changes than either one of them by itself. We show how the algorithm performs on a real surveillance scenario where the face undergoes various pose and illumination changes. The algorithm runs in real-time at 20 fps on a standard 3.0 GHz desktop PC.

[1]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[2]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[3]  Paul A. Viola,et al.  Robust Real-time Object Detection , 2001 .

[4]  M. Alexander,et al.  Principles of Neural Science , 1981 .

[5]  Dorin Comaniciu,et al.  Robust detection and tracking of human faces with an active camera , 2000, Proceedings Third IEEE International Workshop on Visual Surveillance.

[6]  Rama Chellappa,et al.  Visual tracking and recognition using appearance-adaptive models in particle filters , 2004, IEEE Transactions on Image Processing.

[7]  David J. Fleet,et al.  Robust Online Appearance Models for Visual Tracking , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Christoph von der Malsburg,et al.  Tracking and learning graphs and pose on image sequences of faces , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[9]  Quan Pan,et al.  Reliable and fast tracking of faces under varying pose , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[10]  Vijendran G. Venkoparao,et al.  Face Detection by using Skin Color Model based on One Class Classifier , 2006, 9th International Conference on Information Technology (ICIT'06).

[11]  Pawan Sinha,et al.  Receptive Field Structures for Recognition , 2006, Neural Computation.

[12]  Neil J. Gordon,et al.  Editors: Sequential Monte Carlo Methods in Practice , 2001 .

[13]  Michael Isard,et al.  Contour Tracking by Stochastic Propagation of Conditional Density , 1996, ECCV.

[14]  Walter W. Focke,et al.  Mixture Models Based on Neural Network Averaging , 2006, Neural Computation.

[15]  Gary Bradski,et al.  Computer Vision Face Tracking For Use in a Perceptual User Interface , 1998 .