A Hybrid Model Combining Learning Distance Metric and DAG Support Vector Machine for Multimodal Biometric Recognition

Metric learning has significantly improved machine learning applications such as face re-identification and image classification using K-Nearest Neighbor (KNN) and Support Vector Machine (SVM) classifiers. However, to the best of our knowledge, it has not been investigated yet, especially for the multimodal biometric recognition problem in immigration, forensic and surveillance applications with uncontrolled ear datasets. Therefore, it is interesting and very attractive to propose a novel framework for multimodal biometric recognition based on Learning Distance Metric (LDM) via kernel SVM. This paper considers metric learning for SVM by investigating a hybrid Learning Distance Metric and Directed Acyclic Graph SVM (LDM-DAGSVM) model for multimodal biometric recognition, where LDM and DAGSVM are two emerging techniques in dealing with classification problems. Different from existing multimodal biometric recognition methods, the proposed approach aims to learn Mahalanobis distance metric via kernel SVM to maximize the inter-class variations and minimize the intra-class variations, simultaneously. Experimental results on the uncontrolled datasets such as AR face and AWE ear datasets show that the proposed approach achieves competitive performance compared with models working on individual modalities and overperforms the state-of-the-art multimodal methods. The proposed model achieves five-fold classification accuracy around 99.85 % for the face and ear images.

[1]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Bhabatosh Chanda,et al.  Face recognition using fusion of feature learning techniques , 2019, Measurement.

[3]  Gang Xiao,et al.  Discriminative Local Feature Fusion for Ear Recognition Problem , 2018, ICBBB 2018.

[4]  Zhi-Chun Mu,et al.  Multi-modality recognition of human face and ear based on deep learning , 2017, 2017 International Conference on Wavelet Analysis and Pattern Recognition (ICWAPR).

[5]  Mohamed Abdel-Mottaleb,et al.  Discriminant Correlation Analysis: Real-Time Feature Level Fusion for Multimodal Biometric Recognition , 2016, IEEE Transactions on Information Forensics and Security.

[6]  Feiping Nie,et al.  Learning a Mahalanobis distance metric for data clustering and classification , 2008, Pattern Recognit..

[7]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[8]  Peter Peer,et al.  Ear recognition: More than a survey , 2016, Neurocomputing.

[9]  Mohammad H. Mahoor,et al.  Multi-modal ear and face modeling and recognition , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[10]  Slim Essid,et al.  A Multimodal Approach to Speaker Diarization on TV Talk-Shows , 2013, IEEE Transactions on Multimedia.

[11]  Phalguni Gupta,et al.  Multimodal Belief Fusion for Face and Ear Biometrics , 2009, Intell. Inf. Manag..

[12]  Shuicheng Yan,et al.  An HOG-LBP human detector with partial occlusion handling , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[13]  Avinash C. Kak,et al.  PCA versus LDA , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Maurício Pamplona Segundo,et al.  Employing Fusion of Learned and Handcrafted Features for Unconstrained Ear Recognition , 2017, IET Biom..

[15]  Liu Xia,et al.  Face recognition algorithm based on improved kernel sparse representation , 2019, 2019 34rd Youth Academic Annual Conference of Chinese Association of Automation (YAC).

[16]  Usman Saeed,et al.  Combining ear-based traditional and soft biometrics for unconstrained ear recognition , 2018, J. Electronic Imaging.

[17]  Bhabani Shankar Prasad Mishra,et al.  Fusion of PHOG and LDP local descriptors for kernel-based ear biometric recognition , 2018, Multimedia Tools and Applications.

[18]  Jiwen Lu,et al.  Regularized local metric learning for person re-identification , 2015, Pattern Recognit. Lett..

[19]  Cornélia J. P. Passarinho,et al.  Curved Gabor Projection Entropy for Face Recognition , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[20]  Phalguni Gupta,et al.  Efficient human recognition system using ear and profile face , 2013, 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[21]  Nicholas Costen,et al.  Multimodal biometric system for ECG, ear and iris recognition based on local descriptors , 2019, Multimedia Tools and Applications.

[22]  Pinaki Prasad Guha Neogi,et al.  Triangular coil pattern of local radius of gyration face for heterogeneous face recognition , 2019, Applied Intelligence.

[23]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[24]  Martti Juhola,et al.  DAGSVM vs. DAGKNN: An Experimental Case Study with Benthic Macroinvertebrate Dataset , 2012, MLDM.

[25]  Marina L. Gavrilova,et al.  Multimodal Biometric System Using Rank-Level Fusion Approach , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[26]  Xin Li,et al.  Combination recognition of face and ear based on two-dimensional fisher linear discriminant , 2013, 2013 IEEE 4th International Conference on Software Engineering and Service Science.

[27]  M.N.S. Swamy,et al.  Normalization and Weighting Techniques Based on Genuine-Impostor Score Fusion in Multi-Biometric Systems , 2018, IEEE Transactions on Information Forensics and Security.

[28]  Yuan Yan Tang,et al.  Sparse Supervised Representation-Based Classifier for Uncontrolled and Imbalanced Classification , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[29]  Lina J. Karam,et al.  Unconstrained ear recognition using deep neural networks , 2018, IET Biom..

[30]  Yongcun Cao,et al.  Feature fusion in multimodal recognition based on ear and profile face , 2008, International Symposium on Instrumentation and Control Technology.

[31]  Heydi Mendez Vazquez,et al.  Metric Learning in the Dissimilarity Space to Improve Low-Resolution Face Recognition , 2016, CIARP.

[32]  Gang Xiao,et al.  Deep features for efficient multi-biometric recognition with face and ear images , 2017, International Conference on Digital Image Processing.

[33]  Hongzhi Zhang,et al.  Metric Learning with Dynamically Generated Pairwise Constraints for Ear Recognition , 2018, Inf..

[34]  Wangmeng Zuo,et al.  Learning LogDet Divergence for Ear Recognition , 2018, ICBEA '18.

[35]  Arun Ross,et al.  Multimodal biometrics: An overview , 2004, 2004 12th European Signal Processing Conference.

[36]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[37]  Kin-Man Lam,et al.  Multi-resolution feature fusion for face recognition , 2014, Pattern Recognit..

[38]  David Zhang,et al.  A Kernel Classification Framework for Metric Learning , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[39]  Li Zhang,et al.  Wonder ears: Identification of identical twins from ear images , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[40]  Saiful Islam,et al.  Mahalanobis Distance , 2009, Encyclopedia of Biometrics.

[41]  Rameswar Debnath,et al.  A decision based one-against-one method for multi-class support vector machine , 2004, Pattern Analysis and Applications.

[42]  Jian Yang,et al.  Feature fusion: parallel strategy vs. serial strategy , 2003, Pattern Recognit..

[43]  Hong Man,et al.  Face recognition based on multi-class mapping of Fisher scores , 2005, Pattern Recognit..

[44]  Hongyu Li,et al.  3D Ear Identification Using Block-Wise Statistics-Based Features and LC-KSVD , 2016, IEEE Transactions on Multimedia.

[45]  Ester González Sosa,et al.  Face and Body Biometrics in the Wild: Advances in the Visible Spectrum and Beyond , 2017 .

[46]  Yi Zhang,et al.  Ear verification under uncontrolled conditions with convolutional neural networks , 2018, IET Biom..

[47]  Zhi-Chun Mu,et al.  Multimodal recognition based on face and ear , 2007, 2007 International Conference on Wavelet Analysis and Pattern Recognition.

[48]  David Zhang,et al.  Ear-parotic face angle: A unique feature for 3D ear recognition , 2015, Pattern Recognit. Lett..

[49]  Jiwen Lu,et al.  Discriminative Deep Metric Learning for Face and Kinship Verification , 2017, IEEE Transactions on Image Processing.

[50]  Dacheng Tao,et al.  Robust Face Recognition via Multimodal Deep Face Representation , 2015, IEEE Transactions on Multimedia.

[51]  Nello Cristianini,et al.  Large Margin DAGs for Multiclass Classification , 1999, NIPS.

[52]  S. Sathiya Keerthi,et al.  Which Is the Best Multiclass SVM Method? An Empirical Study , 2005, Multiple Classifier Systems.

[53]  Amitava Chatterjee,et al.  Image set based ear recognition using novel dictionary learning and classification scheme , 2016, Eng. Appl. Artif. Intell..

[54]  Sudeep Sarkar,et al.  Comparison and Combination of Ear and Face Images in Appearance-Based Biometrics , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[55]  Chokri Ben Amar,et al.  Local generic representation for patch uLBP-based face recognition with single training sample per subject , 2018, Multimedia Tools and Applications.

[56]  Anil K. Jain,et al.  Hacking Mobile Phones Using 2 D Printed Fingerprints , 2016 .

[57]  Yajie Tian,et al.  Handbook of face recognition , 2003 .

[58]  Abdelmgeid A. Ali,et al.  Ear recognition using local binary patterns: A comparative experimental study , 2019, Expert Syst. Appl..

[59]  Liping Chen,et al.  A robust face and ear based multimodal biometric system using sparse representation , 2013, Pattern Recognit..

[60]  Yi Liu,et al.  One-against-all multi-class SVM classification using reliability measures , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[61]  Dejan Gjorgjevikj,et al.  A Multi-class SVM Classifier Utilizing Binary Decision Tree , 2009, Informatica.

[62]  Latha Lakshmanan,et al.  Efficient person authentication based on multi-level fusion of ear scores , 2013, IET Biom..

[63]  Earnest Eugene Hansley,et al.  Identification of Individuals from Ears in Real World Conditions , 2018 .

[64]  Martti Juhola,et al.  Directed acyclic graph support vector machines in classification of benthic macroinvertebrate samples , 2014, Artificial Intelligence Review.

[65]  Hui Chen,et al.  Human Ear Recognition in 3D , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.