11K Hands: Gender recognition and biometric identification using a large dataset of hand images

Human hand not only possesses distinctive feature for gender information, it is also considered one of the primary biometric traits used to identify a person. Unlike face images, which are usually unconstrained, an advantage of hand images is they are usually captured under a controlled position. Most state-of-the-art methods, that rely on hand images for gender recognition or biometric identification, employ handcrafted features to train an off-the-shelf classifier or be used by a similarity metric for biometric identification. In this work, we propose a deep learning-based method to tackle the gender recognition and biometric identification problems. Specifically, we design a two-stream convolutional neural network (CNN) which accepts hand images as input and predicts gender information from these hand images. This trained model is then used as a feature extractor to feed a set of support vector machine classifiers for biometric identification. As part of this effort, we propose a large dataset of human hand images, 11K Hands, which contains dorsal and palmar sides of human hand images with detailed ground-truth information for different problems including gender recognition and biometric identification. By leveraging thousands of hand images, we could effectively train our CNN-based model achieving promising results. One of our findings is that the dorsal side of human hands is found to have effective distinctive features similar to, if not better than, those available in the palmar side of human hand images. To facilitate access to our 11K Hands dataset, the dataset, the trained CNN models, and our Matlab source code are available at (https://goo.gl/rQJndd).

[1]  Ieee Xplore,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Information for Authors , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Shanmukhappa A. Angadi,et al.  Hand geometry based user identification using minimal edge connected hand image graph , 2018, IET Comput. Vis..

[3]  Michael F. Cohen,et al.  Digital photography with flash and no-flash image pairs , 2004, ACM Trans. Graph..

[4]  Ahmad Sufril Azlan Mohamed,et al.  An overview of hand-based multimodal biometrie system using multi-classifier score fusion with score normalization , 2017, 2017 International Conference on Signal Processing and Communication (ICSPC).

[5]  Joachim M. Buhmann,et al.  Distortion Invariant Object Recognition in the Dynamic Link Architecture , 1993, IEEE Trans. Computers.

[6]  Loris Nanni,et al.  Handcrafted vs. non-handcrafted features for computer vision classification , 2017, Pattern Recognit..

[7]  Alexander Binder,et al.  Understanding and Comparing Deep Neural Networks for Age and Gender Classification , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[8]  Richard Szeliski,et al.  Digital photography with flash and no-flash image pairs , 2004, ACM Trans. Graph..

[9]  Abdallah Meraoumia,et al.  Using of Finger-Knuckle-Print in biometric security systems , 2016, 2016 International Conference on Information Technology for Organizations Development (IT4OD).

[10]  David Zhang,et al.  Online Palmprint Identification , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Jian Sun,et al.  Guided Image Filtering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Yoshihiko Hamamoto,et al.  A gabor filter-based method for recognizing handwritten numerals , 1998, Pattern Recognit..

[13]  Yun Fu,et al.  Bimodal gender recognition from face and fingerprint , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[14]  Mita Nasipuri,et al.  Finger contour profile based hand biometric recognition , 2016, Multimedia Tools and Applications.

[15]  Tanuj Kanchan,et al.  Sex Determination from Hand and Foot Dimensions in a North Indian Population , 2011, Journal of forensic sciences.

[16]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[17]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[18]  K. Bowyer,et al.  Predicting ethnicity and gender from iris texture , 2011, 2011 IEEE International Conference on Technologies for Homeland Security (HST).

[19]  Ying Shen,et al.  Towards contactless palmprint recognition: A novel device, a new benchmark, and a collaborative representation based identification approach , 2017, Pattern Recognit..

[20]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Shiguang Shan,et al.  Occlusion Aware Facial Expression Recognition Using CNN With Attention Mechanism , 2019, IEEE Transactions on Image Processing.

[22]  Arun Ross,et al.  What Else Does Your Biometric Data Reveal? A Survey on Soft Biometrics , 2016, IEEE Transactions on Information Forensics and Security.

[23]  Ming Wu,et al.  Gender Classification Based on Geometry Features of Palm Image , 2014, TheScientificWorldJournal.

[24]  Zeev Farbman,et al.  Edge-preserving decompositions for multi-scale tone and detail manipulation , 2008, ACM Trans. Graph..

[25]  Ayesha Gurnani,et al.  VEGAC: Visual Saliency-based Age, Gender, and Facial Expression Classification Using Convolutional Neural Networks , 2018, ArXiv.

[26]  Andrey Kuehlkamp,et al.  Gender-from-Iris or Gender-from-Mascara? , 2017, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[27]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[28]  David Zhang,et al.  A survey of palmprint recognition , 2009, Pattern Recognit..

[29]  Erin Shubel,et al.  Relative Lengths of Fingers and Toes in Human Males and Females , 2002, Hormones and Behavior.

[30]  Shiv Ram Dubey,et al.  Identity verification using shape and geometry of human hands , 2015, Expert Syst. Appl..

[31]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[32]  Andrew Zisserman,et al.  Return of the Devil in the Details: Delving Deep into Convolutional Nets , 2014, BMVC.

[33]  Koen E. A. van de Sande,et al.  Evaluating Color Descriptors for Object and Scene Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  A. Enis Çetin,et al.  Gender classification based on isolated facial features and foggy faces using jointly trained deep convolutional neural network , 2018, J. Electronic Imaging.

[35]  Shahrel Azmin Suandi,et al.  Finger Vein Recognition Using Local Line Binary Pattern , 2011, Sensors.

[36]  Arun Ross,et al.  Impact of facial cosmetics on automatic gender and age estimation algorithms , 2014, 2014 International Conference on Computer Vision Theory and Applications (VISAPP).

[37]  Adel M. Alimi,et al.  Novel hand biometric system using invariant descriptors , 2014, 2014 6th International Conference of Soft Computing and Pattern Recognition (SoCPaR).

[38]  Baiqiang Xia,et al.  Which Facial Expressions Can Reveal Your Gender? A Study With 3D Faces , 2018, ArXiv.

[39]  Alan F. Smeaton,et al.  Detector adaptation by maximising agreement between independent data sources , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  King-Sun Fu,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Publication Information , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Thomas Mensink,et al.  Image Classification with the Fisher Vector: Theory and Practice , 2013, International Journal of Computer Vision.

[42]  Thomas B. Moeslund,et al.  On soft biometrics , 2015, Pattern Recognit. Lett..

[43]  Chin-Chuan Han A hand-based personal authentication using a coarse-to-fine strategy , 2004, Image Vis. Comput..

[44]  Tieniu Tan,et al.  Demographic Analysis from Biometric Data: Achievements, Challenges, and New Frontiers , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Tieniu Tan,et al.  Ordinal palmprint represention for personal identification [represention read representation] , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[46]  Tanuj Kanchan,et al.  Anthropometry of hand in sex determination of dismembered remains - A review of literature. , 2011, Journal of forensic and legal medicine.

[47]  Anil K. Jain,et al.  Can soft biometric traits assist user recognition? , 2004, SPIE Defense + Commercial Sensing.

[48]  Ajay Kumar,et al.  Incorporating Cohort Information for Reliable Palmprint Authentication , 2008, 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing.

[49]  Junzhou Huang,et al.  Face Landmark Fitting via Optimized Part Mixtures and Cascaded Deformable Model , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Woo Chaw Seng,et al.  A review of biometric technology along with trends and prospects , 2014, Pattern Recognit..

[51]  Arun Ross,et al.  Handbook of Biometrics , 2007 .

[52]  Aly A. Farag,et al.  CSIFT: A SIFT Descriptor with Color Invariant Characteristics , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[53]  Mahmoud Afifi,et al.  AFIF4: Deep Gender Classification based on AdaBoost-based Fusion of Isolated Facial Features and Foggy Faces , 2017, J. Vis. Commun. Image Represent..

[54]  Atsuto Maki,et al.  Factors of Transferability for a Generic ConvNet Representation , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Adel M. Alimi,et al.  Bimodal biometric system for hand shape and palmprint recognition based on SIFT sparse representation , 2017, Multimedia Tools and Applications.

[56]  Debotosh Bhattacharjee,et al.  Human Identification Using Selected Features From Finger Geometric Profiles , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[57]  V. Kecman,et al.  Iterative Single Data Algorithm for Training Kernel Machines from Huge Data Sets: Theory and Performance , 2005 .

[58]  Shiguang Shan,et al.  Patch-Gated CNN for Occlusion-aware Facial Expression Recognition , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[59]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[60]  David Zhang,et al.  A Study of Hand Back Skin Texture Patterns for Personal Identification and Gender Classification , 2012, Sensors.

[61]  P. Duygulu,et al.  Visual categorization with bags of keypoints , 2002, eccv 2002.

[62]  Baochang Zhang,et al.  Local Derivative Pattern Versus Local Binary Pattern: Face Recognition With High-Order Local Pattern Descriptor , 2010, IEEE Transactions on Image Processing.

[63]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[64]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[65]  Mario Baum,et al.  Handbook Of Biometrics , 2016 .

[66]  David Zhang,et al.  Hand shape recognition based on coherent distance shape contexts , 2012, Pattern Recognit..

[67]  Bülent Sankur,et al.  Shape-based hand recognition , 2006, IEEE Transactions on Image Processing.

[68]  Antitza Dantcheva,et al.  Can a Smile Reveal Your Gender? , 2016, 2016 International Conference of the Biometrics Special Interest Group (BIOSIG).

[69]  Ajay Kumar,et al.  Palmprint recognition using rank level fusion , 2010, 2010 IEEE International Conference on Image Processing.

[70]  Mahmoud Afifi,et al.  Can we boost the power of the Viola–Jones face detector using preprocessing? An empirical study , 2018, J. Electronic Imaging.

[71]  Taolue Chen,et al.  Deep finger texture learning for verifying people , 2018, IET Biom..

[72]  Miguel A. Ferrer,et al.  Low Cost Multimodal Biometric identification System Based on Hand Geometry, Palm and Finger Print Texture , 2007, 2007 41st Annual IEEE International Carnahan Conference on Security Technology.

[73]  Rama Chellappa,et al.  HyperFace: A Deep Multi-Task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[74]  Mircea Nicolescu,et al.  Gender classification from hand shape , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[75]  Balasubramanian Raman,et al.  Compass local binary patterns for gender recognition of facial photographs and sketches , 2016, Neurocomputing.