FaceCAPTCHA: a CAPTCHA that identifies the gender of face images unrecognized by existing gender classifiers

Computers tend to fail to classify human faces by gender, especially upon changes in viewpoint or upon occlusion that make it more difficult to extract the necessary image features. In contrast, humans are good at identifying gender but have difficulties in dealing with a large number of images. Accounting for this gap, we proposed FaceCAPTCHA, a novel image-based CAPTCHA that asks users to identify the gender of face images whose gender cannot be recognized by computers (gender-indiscernible faces). By converting the manual gender classification task into a CAPTCHA test, FaceCAPTCHA was designed to not only continuously identify the gender of gender-indiscernible faces but also differentiate between humans and computers and generate new test images. Our user studies showed that FaceCAPTCHA reliably identifies gender-indiscernible faces. A single eight-image FaceCAPTCHA test was completed in 12.41 s on average with a human success rate of 86.51 %, which can be further increased by filtering error-prone test images. In contrast, the probability of passing a FaceCAPTCHA test by random guessing was 0.006 %. We could therefore conclude that FaceCAPTCHA is robust against malicious attacks and easy enough for practical use.

[1]  Wenze Li,et al.  Face Detection in Complex Background Using AdaBoost Algorithm , 2012, 2012 Sixth International Conference on Internet Computing for Science and Engineering.

[2]  Manuel Blum,et al.  Improving Image Search with PHETCH , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[3]  Tetsunori Kobayashi,et al.  A method of gender classification by integrating facial, hairstyle, and clothing images , 2004, ICPR 2004.

[4]  Benjamin B. Bederson,et al.  Human computation: a survey and taxonomy of a growing field , 2011, CHI.

[5]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[6]  Rich Gossweiler,et al.  WWW 2009 MADRID! Track: User Interfaces and Mobile Web / Session: User Interfaces What’s Up CAPTCHA? A CAPTCHA Based on Image Orientation , 2022 .

[7]  Eli Biham,et al.  Advances in Cryptology — EUROCRYPT 2003 , 2003, Lecture Notes in Computer Science.

[8]  Abdul Rehman Abbasi,et al.  An Image-Based CAPTCHA Scheme Exploiting Human Appearance Characteristics , 2012, KSII Trans. Internet Inf. Syst..

[9]  Ji Zheng,et al.  A support vector machine classifier with automatic confidence and its application to gender classification , 2011, Neurocomputing.

[10]  J. Doug Tygar,et al.  Image Recognition CAPTCHAs , 2004, ISC.

[11]  Jeff Z. Pan,et al.  The Semanic Web: Research and Applications - 8th Extended Semantic Web Conference, ESWC 2011, Heraklion, Crete, Greece, May 29 - June 2, 2011, Proceedings, Part II , 2011, ESWC.

[12]  Zicheng Liu,et al.  ARTiFACIAL: automated reverse turing test using FACIAL features , 2003, MULTIMEDIA '03.

[13]  Mor Naaman,et al.  Context data in geo-referenced digital photo collections , 2004, MULTIMEDIA '04.

[14]  Hwan-Gue Cho,et al.  A new image-based CAPTCHA using the orientation of the polygonally cropped sub-images , 2010, The Visual Computer.

[15]  Nuno Correia,et al.  A gesture based game for image tagging , 2008, CHI Extended Abstracts.

[16]  Mario Köppen,et al.  Advances in Neuro-Information Processing, 15th International Conference, ICONIP 2008, Auckland, New Zealand, November 25-28, 2008, Revised Selected Papers, Part I , 2009, International Conference on Neural Information Processing.

[17]  Guodong Guo,et al.  Gender from Body: A Biologically-Inspired Approach with Manifold Learning , 2009, ACCV.

[18]  Jon Howell,et al.  Asirra: a CAPTCHA that exploits interest-aligned manual image categorization , 2007, CCS '07.

[19]  Mor Naaman,et al.  HT06, tagging paper, taxonomy, Flickr, academic article, to read , 2006, HYPERTEXT '06.

[20]  Luís A. Alexandre.  Gender recognition: A multiscale decision fusion approach , 2010, Pattern Recognit. Lett..

[21]  Manuel Blum,et al.  reCAPTCHA: Human-Based Character Recognition via Web Security Measures , 2008, Science.

[22]  Hongbin Zha,et al.  Computer Vision - ACCV 2009, 9th Asian Conference on Computer Vision, Xi'an, China, September 23-27, 2009, Revised Selected Papers, Part III , 2010, Asian Conference on Computer Vision.

[23]  Kris Gaj,et al.  Face Recognition CAPTCHAs , 2006, Advanced Int'l Conference on Telecommunications and Int'l Conference on Internet and Web Applications and Services (AICT-ICIW'06).

[24]  Caifeng Shan,et al.  Learning local binary patterns for gender classification on real-world face images , 2012, Pattern Recognit. Lett..

[25]  Tetsunori Kobayashi,et al.  A method of gender classification by integrating facial, hairstyle, and clothing images , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[26]  Shree K. Nayar,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence Describable Visual Attributes for Face Verification and Image Search , 2022 .

[27]  D. Perrett,et al.  What Gives a Face its Gender? , 1993, Perception.

[28]  Paul A. Viola,et al.  A unified learning framework for real time face detection and classification , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[29]  Bao-Liang Lu,et al.  Gender Classification by Combining Facial and Hair Information , 2009, ICONIP.

[30]  Elena Paslaru Bontas Simperl,et al.  SeaFish: A Game for Collaborative and Visual Image Annotation and Interlinking , 2011, ESWC.

[31]  Deva Ramanan,et al.  Face detection, pose estimation, and landmark localization in the wild , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Chao Yang,et al.  Attacks and design of image recognition CAPTCHAs , 2010, CCS '10.

[33]  John Langford,et al.  CAPTCHA: Using Hard AI Problems for Security , 2003, EUROCRYPT.

[34]  Markus Jakobsson,et al.  Making CAPTCHAs clickable , 2008, HotMobile '08.

[35]  Laura A. Dabbish,et al.  Labeling images with a computer game , 2004, AAAI Spring Symposium: Knowledge Collection from Volunteer Contributors.

[36]  José Miguel Buenaposada,et al.  Revisiting Linear Discriminant Techniques in Gender Recognition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Matthew Toews,et al.  Detection, Localization, and Sex Classification of Faces from Arbitrary Viewpoints and under Occlusion , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Manuel Blum,et al.  Peekaboom: a game for locating objects in images , 2006, CHI.

[39]  Tal Hassner,et al.  Similarity Scores Based on Background Samples , 2009, ACCV.

[40]  Mary Czerwinski,et al.  Designing human friendly human interaction proofs (HIPs) , 2005, CHI.

[41]  Eric Bruno,et al.  TagCaptcha: annotating images with CAPTCHAs , 2009, HCOMP '09.

[42]  Antonio Torralba,et al.  LabelMe: A Database and Web-Based Tool for Image Annotation , 2008, International Journal of Computer Vision.