Meta-Mining Discriminative Samples for Kinship Verification

Kinship verification aims to find out whether there is a kin relation for a given pair of facial images. Kinship verification databases are born with unbalanced data. For a database with N positive kinship pairs, we naturally obtain N(N − 1) negative pairs. How to fully utilize the limited positive pairs and mine discriminative information from sufficient negative samples for kinship verification remains an open issue. To address this problem, we propose a Discriminative Sample Meta-Mining (DSMM) approach in this paper. Unlike existing methods that usually construct a balanced dataset with fixed negative pairs, we propose to utilize all possible pairs and automatically learn discriminative information from data. Specifically, we sample an unbalanced train batch and a balanced meta-train batch for each iteration. Then we learn a meta-miner with the meta-gradient on the balanced meta-train batch. In the end, the samples in the unbalanced train batch are re-weighted by the learned meta-miner to optimize the kinship models. Experimental results on the widely used KinFaceW-I, KinFaceW-II, TSKinFace, and Cornell Kinship datasets demonstrate the effectiveness of the proposed approach.

[1]  Ming Tang,et al.  Adaptive Variance Based Label Distribution Learning for Facial Age Estimation , 2020, ECCV.

[2]  Jiwen Lu,et al.  Graph-Based Social Relation Reasoning , 2020, ECCV.

[3]  Hao Pan,et al.  IDA-3D: Instance-Depth-Aware 3D Object Detection From Stereo Vision for Autonomous Driving , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Nicolas Usunier,et al.  End-to-End Object Detection with Transformers , 2020, ECCV.

[5]  Jie Zhou,et al.  Graph-Based Kinship Reasoning Network , 2020, 2020 IEEE International Conference on Multimedia and Expo (ICME).

[6]  Shaodi You,et al.  Kinship Identification through Joint Learning Using Kinship Verification Ensemble , 2020, ECCV.

[7]  Feiyue Huang,et al.  CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognition , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Dong Cao,et al.  Domain Balancing: Face Recognition on Long-Tailed Domains , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Dong Cao,et al.  Learning Meta Face Recognition in Unseen Domains , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Jianfei Yang,et al.  Suppressing Uncertainties for Large-Scale Facial Expression Recognition , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Xiu-Shen Wei,et al.  BBN: Bilateral-Branch Network With Cumulative Learning for Long-Tailed Visual Recognition , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Maja Pantic,et al.  Efficient N-Dimensional Convolutions via Higher-Order Factorization , 2019, ArXiv.

[13]  Jiwen Lu,et al.  BridgeNet: A Continuity-Aware Probabilistic Network for Age Estimation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Qinghua Hu,et al.  Weighted Graph Embedding-Based Metric Learning for Kinship Verification , 2019, IEEE Transactions on Image Processing.

[15]  Qi Xie,et al.  Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting , 2019, NeurIPS.

[16]  Yang Song,et al.  Class-Balanced Loss Based on Effective Number of Samples , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Ming Shao,et al.  Visual Kinship Recognition of Families in the Wild , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Bin Yang,et al.  Learning to Reweight Examples for Robust Deep Learning , 2018, ICML.

[19]  Ling Shao,et al.  Learning a Multiple Kernel Similarity Metric for kinship verification , 2018, Inf. Sci..

[20]  Stefanos Zafeiriou,et al.  ArcFace: Additive Angular Margin Loss for Deep Face Recognition , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  F. Xavier Roca,et al.  Age and gender recognition in the wild with deep attention , 2017, Pattern Recognit..

[22]  Tao Xiang,et al.  Learning to Compare: Relation Network for Few-Shot Learning , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[23]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[24]  Pieter Abbeel,et al.  A Simple Neural Attentive Meta-Learner , 2017, ICLR.

[25]  Jiwen Lu,et al.  Discriminative Deep Metric Learning for Face and Kinship Verification , 2017, IEEE Transactions on Image Processing.

[26]  Razvan Pascanu,et al.  A simple neural network module for relational reasoning , 2017, NIPS.

[27]  Bhiksha Raj,et al.  SphereFace: Deep Hypersphere Embedding for Face Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Richard S. Zemel,et al.  Prototypical Networks for Few-shot Learning , 2017, NIPS.

[29]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[30]  Abdenour Hadid,et al.  On the usefulness of color for kinship verification from face images , 2016, 2016 IEEE International Workshop on Information Forensics and Security (WIFS).

[31]  Misha Denil,et al.  Learning to Learn without Gradient Descent by Gradient Descent , 2016, ICML.

[32]  Hugo Larochelle,et al.  Optimization as a Model for Few-Shot Learning , 2016, ICLR.

[33]  Daan Wierstra,et al.  Meta-Learning with Memory-Augmented Neural Networks , 2016, ICML.

[34]  Marcin Andrychowicz,et al.  Learning to learn by gradient descent by gradient descent , 2016, NIPS.

[35]  Oriol Vinyals,et al.  Matching Networks for One Shot Learning , 2016, NIPS.

[36]  Qingming Huang,et al.  Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networks , 2015, ECCV.

[37]  Jiwen Lu,et al.  Prototype-Based Discriminative Feature Learning for Kinship Verification , 2015, IEEE Transactions on Cybernetics.

[38]  Venkatesh Saligrama,et al.  Group Membership Prediction , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[39]  Xiaoou Tang,et al.  Learning Social Relation Traits from Face Images , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[40]  Tal Hassner,et al.  Age and gender classification using convolutional neural networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[41]  Songcan Chen,et al.  Tri-Subject Kinship Verification: Understanding the Core of A Family , 2015, IEEE Transactions on Multimedia.

[42]  Jiwen Lu,et al.  Large Margin Multi-metric Learning for Face and Kinship Verification in the Wild , 2014, ACCV.

[43]  Longbing Cao,et al.  Social Image Analysis From a Non-IID Perspective , 2014, IEEE Transactions on Multimedia.

[44]  Jiwen Lu,et al.  Discriminative Multimetric Learning for Kinship Verification , 2014, IEEE Transactions on Information Forensics and Security.

[45]  Afshin Dehghan,et al.  Who Do I Look Like? Determining Parent-Offspring Resemblance via Gated Autoencoders , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Chandra Kambhamettu,et al.  Can faces verify blood-relations? , 2012, 2012 IEEE Fifth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[47]  Richa Singh,et al.  Self-similarity representation of Weber faces for kinship classification , 2012, 2012 IEEE Fifth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[48]  Jiwen Lu,et al.  Gabor-based gradient orientation pyramid for kinship verification under uncontrolled environments , 2012, ACM Multimedia.

[49]  Jiebo Luo,et al.  Understanding Kin Relationships in a Photo , 2012, IEEE Transactions on Multimedia.

[50]  David W. Jacobs,et al.  Generalized Multiview Analysis: A discriminative latent space , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[51]  Jiwen Lu,et al.  Neighborhood repulsed metric learning for kinship verification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[52]  Jiwen Lu,et al.  Kinship verification from facial images under uncontrolled conditions , 2011, ACM Multimedia.

[53]  Tsuhan Chen,et al.  Towards computational models of kinship verification , 2010, 2010 IEEE International Conference on Image Processing.

[54]  Kai Ming Ting,et al.  A Comparative Study of Cost-Sensitive Boosting Algorithms , 2000, ICML.

[55]  Seetha Hari,et al.  Learning From Imbalanced Data , 2019, Advances in Computer and Electrical Engineering.

[56]  Richa Singh,et al.  Hierarchical Representation Learning for Kinship Verification , 2017, IEEE Transactions on Image Processing.

[57]  Liang Wang,et al.  Kinship Verification with Deep Convolutional Neural Networks , 2015, BMVC.