An efficient IP approach to constrained multiple face tracking and recognition

Tracking and recognition of objects, such as faces, in video is commonly accomplished in independent fashion. However, important information is contained in both problems that could be used to increase the overall recognition accuracy. We propose a unified integer program (IP) based framework for multi-object tracking and recognition in video, where the two tasks are conducted jointly, using a set of natural constraints. In the domain of multiple face recognition, pairing constraints limit the number of objects that can be labeled with the same identity while temporal constraints allow the important information about objects identities's to be used to improve tracking. Despite its appeal, the solving the IP objective can be inefficient in real-world scenarios. For this reason, we employ an approximate Generalized Assignment Problem (GAP) solution to the IP problem, which is both theoretically appealing and computationally highly efficient. We finally demonstrate that the IP and GAP methods of conducting multi-object tracking and recognition can be successfully applied to real world videos where the traditional methods of conducting tracking and recognition separately fail to produce satisfactory results.

[1]  Jean-Marc Odobez,et al.  Using particles to track varying numbers of interacting people , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[2]  F. Fleuret,et al.  Multiple object tracking using flow linear programming , 2009, 2009 Twelfth IEEE International Workshop on Performance Evaluation of Tracking and Surveillance.

[3]  Zihan Zhou,et al.  Towards a practical face recognition system: Robust registration and illumination by sparse representation , 2009, CVPR.

[4]  Maja Pantic,et al.  Facial point detection using boosted regression and graph models , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  Ramakant Nevatia,et al.  Global data association for multi-object tracking using network flows , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Vladimir Kolmogorov,et al.  Optimizing Binary MRFs via Extended Roof Duality , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Rong Yan,et al.  Multiple instance learning for labeling faces in broadcasting news video , 2005, MULTIMEDIA '05.

[8]  James J. Little,et al.  A Linear Programming Approach for Multiple Object Tracking , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  John E. Beasley,et al.  Heuristic algorithms for the unconstrained binary quadratic programming problem , 1998 .

[10]  Andrew Zisserman,et al.  Hello! My name is... Buffy'' -- Automatic Naming of Characters in TV Video , 2006, BMVC.

[11]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, Comb..

[12]  Konrad Schindler,et al.  Globally Optimal Multi-target Tracking on a Hexagonal Lattice , 2010, ECCV.

[13]  Vladimir Pavlovic,et al.  Face tracking and recognition with visual constraints in real-world videos , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Alex Zelinsky,et al.  Learning OpenCV---Computer Vision with the OpenCV Library (Bradski, G.R. et al.; 2008)[On the Shelf] , 2009, IEEE Robotics & Automation Magazine.

[15]  Reuven Cohen,et al.  An efficient approximation for the Generalized Assignment Problem , 2006, Inf. Process. Lett..

[16]  Q. Zhang.  A NEW POLYNOMIAL-TIME ALGORITHM FOR LP , 1996 .

[17]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Cordelia Schmid,et al.  Automatic face naming with caption-based supervision , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  M. Saquib Sarfraz,et al.  Probabilistic learning for fully automatic face recognition across pose , 2010, Image Vis. Comput..