Adversarial Similarity Metric Learning for Kinship Verification

Given a pair of facial images, it is an interesting yet challenging problem to determine if there is a kin relation between them. Recent research on that topic has made encouraging progress by learning a kin similarity metric from kinship data. However, most of the existing metric learning algorithms cannot handle hard samples very well, i.e., some ambiguous test pairs cannot be well classified due to some compounding factors, such as the large age gap or gender difference between the parents and children. To address this, we propose an Adversarial Similarity Metric Learning (ASML) method in this paper. More specifically, ASML consists of two adversarial phases: confusion and discrimination. In confusion phase, ambiguous adversarial pairs are automatically generated to challenge the learned similarity metric; while in discrimination phase, the learned metric tries its best to adjust itself to distinguish both the original pairs and the generated adversarial pairs. Consequently, a robust and discriminative similarity metric can be learned by iteratively performing the two adversarial phases. Experiments on the two widely used kinship datasets demonstrate the efficacy of our proposed ASML method in comparison with the state-of-the-art metric learning solutions to kinship verification.

[1]  Jun Guo,et al.  Extended SRC: Undersampled Face Recognition via Intraclass Variant Dictionary , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  L. Maloney,et al.  Lateralization of kin recognition signals in the human face. , 2010, Journal of vision.

[3]  Jiebo Luo,et al.  Understanding Kin Relationships in a Photo , 2012, IEEE Transactions on Multimedia.

[4]  Ming Shao,et al.  Kinship Verification through Transfer Learning , 2011, IJCAI.

[5]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[7]  Susan C. Roberts,et al.  Kin recognition signals in adult faces , 2009, Vision Research.

[8]  Xiaoyang Tan,et al.  Tri-subjects kinship verification: Understanding the core of a family , 2015, 2015 14th IAPR International Conference on Machine Vision Applications (MVA).

[9]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[10]  Guodong Guo,et al.  Visually Interpretable Representation Learning for Depression Recognition from Facial Images , 2020, IEEE Transactions on Affective Computing.

[11]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Samy Bengio,et al.  Large Scale Online Learning of Image Similarity through Ranking , 2009, IbPRIA.

[13]  Gert R. G. Lanckriet,et al.  Robust Structural Metric Learning , 2013, ICML.

[14]  Xiaolong Wang,et al.  Kinship Measurement on Salient Facial Features , 2012, IEEE Transactions on Instrumentation and Measurement.

[15]  Xiang Li,et al.  Adversarial Metric Learning , 2018, IJCAI.

[16]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[17]  Jiwen Lu,et al.  Kinship verification from facial images under uncontrolled conditions , 2011, ACM Multimedia.

[18]  Guodong Guo,et al.  Ensemble similarity learning for kinship verification from facial images in the wild , 2016, Inf. Fusion.

[19]  Wen Gao,et al.  Manifold-Manifold Distance with application to face recognition based on image set , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  S. Dridi,et al.  Human ability to detect kinship in strangers' faces: effects of the degree of relatedness , 2009, Proceedings of the Royal Society B: Biological Sciences.

[21]  Haibin Yan,et al.  Learning discriminative compact binary face descriptor for kinship verification , 2019, Pattern Recognit. Lett..

[22]  Qinghua Hu,et al.  Weighted Graph Embedding-Based Metric Learning for Kinship Verification , 2019, IEEE Transactions on Image Processing.

[23]  Jiwen Lu,et al.  Discriminative Deep Metric Learning for Face Verification in the Wild , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Jiwen Lu,et al.  The FG 2015 Kinship Verification in the Wild Evaluation , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[25]  A. Alvergne,et al.  Cross-cultural perceptions of facial resemblance between kin. , 2009, Journal of vision.

[26]  Feiping Nie,et al.  Multi-View Clustering and Feature Learning via Structured Sparsity , 2013, ICML.

[27]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[28]  Atilla Baskurt,et al.  Triangular similarity metric learning for face verification , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[29]  Richa Singh,et al.  Supervised Mixed Norm Autoencoder for Kinship Verification in Unconstrained Videos , 2018, IEEE Transactions on Image Processing.

[30]  Guodong Guo,et al.  Learning deep compact similarity metric for kinship verification from face images , 2019, Inf. Fusion.

[31]  Jiwen Lu,et al.  Gabor-based gradient orientation pyramid for kinship verification under uncontrolled environments , 2012, ACM Multimedia.

[32]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[33]  Jiwen Lu,et al.  Neighborhood repulsed metric learning for kinship verification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Yu Zhang,et al.  End-to-End Adversarial Memory Network for Cross-domain Sentiment Classification , 2017, IJCAI.

[35]  Shiguang Shan,et al.  Side-Information based Linear Discriminant Analysis for Face Recognition , 2011, BMVC.

[36]  Jiwen Lu,et al.  Prototype-Based Discriminative Feature Learning for Kinship Verification , 2015, IEEE Transactions on Cybernetics.

[37]  Min Xu,et al.  Kinship Measurement on Face Images by Structured Similarity Fusion , 2016, IEEE Access.

[38]  Ming Shao,et al.  Visual Kinship Recognition of Families in the Wild , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Jiwen Lu,et al.  Discriminative Multimetric Learning for Kinship Verification , 2014, IEEE Transactions on Information Forensics and Security.

[40]  Albert Ali Salah,et al.  Like Father, Like Son: Facial Expression Dynamics for Kinship Verification , 2013, 2013 IEEE International Conference on Computer Vision.

[41]  Gwenaël Kaminski,et al.  Firstborns’ Disadvantage in Kinship Detection , 2010, Psychological science.

[42]  Haibin Yan,et al.  Kinship verification from facial images by scalable similarity fusion , 2016, Neurocomputing.

[43]  Michael I. Jordan,et al.  Distance Metric Learning with Application to Clustering with Side-Information , 2002, NIPS.

[44]  Tsuhan Chen,et al.  Towards computational models of kinship verification , 2010, 2010 IEEE International Conference on Image Processing.

[45]  Min Xu,et al.  Kinship Verification Using Facial Images by Robust Similarity Learning , 2016 .

[46]  Haijun Liu,et al.  Status-aware projection metric learning for kinship verification , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[47]  Afshin Dehghan,et al.  Who Do I Look Like? Determining Parent-Offspring Resemblance via Gated Autoencoders , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Alessandro Ferrero,et al.  Camera as the instrument: the rising trend of vision based measurement , 2014, IEEE Instrumentation & Measurement Magazine.

[49]  Kaizhu Huang,et al.  Sparse Metric Learning via Smooth Optimization , 2009, NIPS.