Augmented faces

In this paper we present a fully automatic system for face augmentation on mobile devices. A user can point his mobile phone to a person and the system recognizes his or her face. A tracking algorithm overlays information about the identified person on the screen, thereby achieving an augmented reality effect. The tracker is running on the mobile client, while the recognition is running on a server. The database on the server is built by a fully autonomous crawling method, which taps social networks. For this work we collected 300 000 images from Facebook. The social context gained during this social network analysis is also used to improve the face recognition. The complete system runs in real time on a state-of-the-art mobile phone and is fully automatic, from offline crawling up to augmentation on the mobile device. It can be used to display more information about the identified persons or as a user interface for mixed reality application. To the best of our knowledge this is the first work, which covers such a system end-to-end.

[1]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[2]  Mor Naaman,et al.  Leveraging context to resolve identity in photo albums , 2005, Proceedings of the 5th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL '05).

[3]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[4]  Mor Naaman,et al.  Why we tag: motivations for annotation in mobile and online media , 2007, CHI.

[5]  Ramesh C. Jain,et al.  Automatic Person Annotation of Family Photo Album , 2006, CIVR.

[6]  Nicolas Pinto,et al.  Beyond simple features: A large-scale feature search approach to unconstrained face recognition , 2011, Face and Gesture 2011.

[7]  Luc Van Gool,et al.  Unsupervised face alignment by robust nonrigid mapping , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[8]  Andrew Zisserman,et al.  Taking the bite out of automated naming of characters in TV video , 2009, Image Vis. Comput..

[9]  David Cox,et al.  Scaling up biologically-inspired computer vision: A case study in unconstrained face recognition on facebook , 2011, CVPR 2011 WORKSHOPS.

[10]  Daniel P. Huttenlocher,et al.  Pictorial Structures for Object Recognition , 2004, International Journal of Computer Vision.

[11]  Yee Whye Teh,et al.  Names and faces in the news , 2004, CVPR 2004.

[12]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[13]  Timothy F. Cootes,et al.  Real-Time Facial Feature Tracking on a Mobile Device , 2011, International Journal of Computer Vision.

[14]  Andrea F. Abate,et al.  2D and 3D face recognition: A survey , 2007, Pattern Recognit. Lett..

[15]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[17]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[18]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[19]  Roberto Brunelli,et al.  Face Recognition: Features Versus Templates , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Luc Van Gool,et al.  Server-side object recognition and client-side object tracking for mobile augmented reality , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[21]  Trevor Darrell,et al.  Toward Large-Scale Face Recognition Using Social Network Context , 2010, Proceedings of the IEEE.

[22]  Andrew Zisserman,et al.  “Who are you?” - Learning person specific classifiers from video , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Yee Whye Teh,et al.  Names and faces in the news , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[24]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[25]  Trevor Darrell,et al.  Autotagging Facebook: Social network context improves photo annotation , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[26]  Barbara Caputo,et al.  Who's Doing What: Joint Modeling of Names and Verbs for Simultaneous Face and Pose Annotation , 2009, NIPS.

[27]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[28]  Ronald Poppe.  Scalable face labeling in online social networks , 2011, Face and Gesture 2011.

[29]  Andrew Zisserman,et al.  Hello! My name is... Buffy'' -- Automatic Naming of Characters in TV Video , 2006, BMVC.