Facial attributes for active authentication on mobile devices

We present a method using facial attributes for continuous authentication of smartphone users. We train a bunch of binary attribute classifiers which provide compact visual descriptions of faces. The learned classifiers are applied to the image of the current user of a mobile device to extract the attributes and then authentication is done by simply comparing the calculated attributes with the enrolled attributes of the original user. Extensive experiments on two publicly available unconstrained mobile face video datasets show that our method is able to capture meaningful attributes of faces and performs better than the previously proposed LBP-based authentication method. We also provide a practical variant of our method for efficient continuous authentication on an actual mobile device by doing extensive platform evaluations of memory usage, power consumption, and authentication speed. Display Omitted Facial attributes are effective for continuous authentication on mobile devices.Attribute-based features are more robust than the low-level ones for authentication.Fusion of attribute-based and low-level features gives the best result.The proposed approach allows fast and energy efficient enrollment and authentication.

[1]  Rogério Schmidt Feris,et al.  Attribute-based people search in surveillance environments , 2009, 2009 Workshop on Applications of Computer Vision (WACV).

[2]  Anil K. Jain,et al.  Soft Biometric Traits for Continuous User Authentication , 2010, IEEE Transactions on Information Forensics and Security.

[3]  Wen Gao,et al.  Local Gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[4]  James Ze Wang,et al.  Content-based image retrieval: approaches and trends of the new age , 2005, MIR '05.

[5]  Jason Thornton,et al.  Person attribute search for large-area video surveillance , 2011, 2011 IEEE International Conference on Technologies for Homeland Security (HST).

[6]  Sandeep Kumar,et al.  Continuous Verification Using Multimodal Biometrics , 2007, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Norbert Krüger,et al.  Face Recognition by Elastic Bunch Graph Matching , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Anil K. Jain,et al.  Integrating Faces, Fingerprints, and Soft Biometric Traits for User Recognition , 2004, ECCV Workshop BioAW.

[9]  Sandeep Kumar,et al.  Using Continuous Face Verification to Improve Desktop Security , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[10]  Peter N. Belhumeur,et al.  POOF: Part-Based One-vs.-One Features for Fine-Grained Categorization, Face Verification, and Attribute Estimation , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Anil K. Jain,et al.  Continuous authentication of mobile user: Fusion of face image and inertial Measurement Unit data , 2015, 2015 International Conference on Biometrics (ICB).

[12]  Christoph Busch,et al.  Unobtrusive User-Authentication on Mobile Phones Using Biometric Gait Recognition , 2010, 2010 Sixth International Conference on Intelligent Information Hiding and Multimedia Signal Processing.

[13]  Silvio Savarese,et al.  Recognizing human actions by attributes , 2011, CVPR 2011.

[14]  Rama Chellappa,et al.  Domain adaptive sparse representation-based classification , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[15]  Sébastien Marcel,et al.  The 2013 speaker recognition evaluation in mobile environment , 2013, 2013 International Conference on Biometrics (ICB).

[16]  P.Susan Lalitha Grace,et al.  Active Authentication on Mobile Devices via Stylometry, Application Usage, Web Browsing, and GPS Location , 2017 .

[17]  Tao Feng,et al.  Continuous mobile authentication using touchscreen gestures , 2012, 2012 IEEE Conference on Technologies for Homeland Security (HST).

[18]  Ali Farhadi,et al.  Describing objects by their attributes , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Bernd Girod,et al.  Outdoors augmented reality on mobile phone using loxel-based visual feature organization , 2008, MIR '08.

[20]  Rajesh Kumar,et al.  Context-Aware Active Authentication Using Smartphone Accelerometer Measurements , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[21]  Anil K. Jain,et al.  Suspect identification based on descriptive facial attributes , 2014, IEEE International Joint Conference on Biometrics.

[22]  Josephine Sullivan,et al.  One millisecond face alignment with an ensemble of regression trees , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  J. Koenderink Q… , 2014, Les noms officiels des communes de Wallonie, de Bruxelles-Capitale et de la communaute germanophone.

[24]  Mohamed Daoudi,et al.  Image indexing & retrieval using intermediate features , 2001, MULTIMEDIA '01.

[25]  Andrew Zisserman,et al.  Learning Visual Attributes , 2007, NIPS.

[26]  Matti Pietikäinen,et al.  Face and Eye Detection for Person Authentication in Mobile Phones , 2007, 2007 First ACM/IEEE International Conference on Distributed Smart Cameras.

[27]  James Hays,et al.  SUN attribute database: Discovering, annotating, and recognizing scene attributes , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Paul A. Viola,et al.  Detecting Pedestrians Using Patterns of Motion and Appearance , 2005, International Journal of Computer Vision.

[29]  ZissermanAndrew,et al.  The Pascal Visual Object Classes Challenge , 2015 .

[30]  Anil K. Jain,et al.  Soft Biometric Traits for Personal Recognition Systems , 2004, ICBA.

[31]  Shree K. Nayar,et al.  Attribute and simile classifiers for face verification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[32]  Shaogang Gong,et al.  Re-id: Hunting Attributes in the Wild , 2014, BMVC.

[33]  Davis E. King,et al.  Dlib-ml: A Machine Learning Toolkit , 2009, J. Mach. Learn. Res..

[34]  Rama Chellappa,et al.  Robust multimodal recognition via multitask multivariate low-rank representations , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[35]  Daoqiang Zhang,et al.  Attribute relation learning for zero-shot classification , 2014, Neurocomputing.

[36]  Rama Chellappa,et al.  Face-based Active Authentication on mobile devices , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[37]  Anil K. Jain,et al.  Face Matching and Retrieval Using Soft Biometrics , 2010, IEEE Transactions on Information Forensics and Security.

[38]  Subhransu Maji,et al.  Describing people: A poselet-based approach to attribute classification , 2011, 2011 International Conference on Computer Vision.

[39]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[40]  Steven P. Weber,et al.  Active Authentication on Mobile Devices via Stylometry, Application Usage, Web Browsing, and GPS Location , 2017, IEEE Systems Journal.

[41]  Gregory R. Ganger,et al.  Secure Continuous Biometric-Enhanced Authentication , 2000 .

[42]  Ying Liu,et al.  A survey of content-based image retrieval with high-level semantics , 2007, Pattern Recognit..

[43]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[45]  Rama Chellappa,et al.  Touch Gesture-Based Active User Authentication Using Dictionaries , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[46]  M. Turk,et al.  Temporal Integration for Continuous Multimodal Biometrics , 2003 .

[47]  Rama Chellappa,et al.  Continuous User Authentication on Mobile Devices: Recent progress and remaining challenges , 2016, IEEE Signal Processing Magazine.

[48]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[49]  Stefanos Zafeiriou,et al.  Robust Discriminative Response Map Fitting with Constrained Local Models , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[50]  Michael K. Reiter,et al.  Password hardening based on keystroke dynamics , 2002, International Journal of Information Security.

[51]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[52]  Cassandra M. Carrillo CONTINUOUS BIOMETRIC AUTHENTICATION FOR AUTHORIZED AIRCRAFT PERSONNEL: A PROPOSED DESIGN , 2003 .

[53]  Anil K. Jain,et al.  Continuous user authentication using temporal information , 2010, Defense + Commercial Sensing.

[54]  Matti Pietikäinen,et al.  Bi-Modal Person Recognition on a Mobile Phone: Using Mobile Phone Data , 2012, 2012 IEEE International Conference on Multimedia and Expo Workshops.

[55]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[56]  Rama Chellappa,et al.  Discriminant analysis of principal components for face recognition , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[57]  Nicolas Pinto,et al.  Beyond simple features: A large-scale feature search approach to unconstrained face recognition , 2011, Face and Gesture 2011.

[58]  Dawn Xiaodong Song,et al.  Touchalytics: On the Applicability of Touchscreen Input as a Behavioral Biometric for Continuous Authentication , 2012, IEEE Transactions on Information Forensics and Security.

[59]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[60]  Shaogang Gong,et al.  Person Re-identification by Attributes , 2012, BMVC.

[61]  Christoph H. Lampert,et al.  Attribute-Based Classification for Zero-Shot Visual Object Categorization , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[62]  Lei Yang,et al.  Accurate online power estimation and automatic battery behavior based power model generation for smartphones , 2010, 2010 IEEE/ACM/IFIP International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS).

[63]  Liyan Zhang,et al.  Context-assisted face clustering framework with human-in-the-loop , 2014, International Journal of Multimedia Information Retrieval.

[64]  Shree K. Nayar,et al.  FaceTracer: A Search Engine for Large Collections of Images with Faces , 2008, ECCV.

[65]  Trevor Darrell,et al.  PANDA: Pose Aligned Networks for Deep Attribute Modeling , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[66]  Sharath Pankanti,et al.  BIOMETRIC IDENTIFICATION , 2000 .