Point-to-Set Similarity Based Deep Metric Learning for Offline Signature Verification

Offline signature verification is a challenging task, where the scarcity of the signature data per writer makes it a few-shot problem. We found that previous deep metric learning based methods, whether in pairs or triplets, are unaware of intra-writer variations and have low training efficiency because only point-to-point (P2P) distances are considered. To address this issue, we present a novel point-to-set (P2S) metric for offline signature verification in this paper. By dividing a training batch into a support set and a query set, our optimization goal is to pull each query to its belonging support set. To further strengthen the P2S metric, a hard mining scheme and a margin strategy are introduced. Experiments conducted on three datasets show the effectiveness of our proposed method.

[1]  Miguel Angel Ferrer-Ballester,et al.  On-line signature recognition through the combination of real dynamic data and synthetically generated static data , 2015, Pattern Recognit..

[2]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Youcef Chibani,et al.  The effective use of the one-class SVM classifier for handwritten signature verification based on writer-independent parameters , 2015, Pattern Recognit..

[4]  Xing Ji,et al.  CosFace: Large Margin Cosine Loss for Deep Face Recognition , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[5]  Qi Qian,et al.  SoftTriple Loss: Deep Metric Learning Without Triplet Sampling , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[6]  Manabu Okawa,et al.  Offline Signature Verification with VLAD Using Fused KAZE Features from Foreground and Background Signature Images , 2017, 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR).

[7]  Benjamin Graham,et al.  Fractional Max-Pooling , 2014, ArXiv.

[8]  Ilias Theodorakopoulos,et al.  Offline Handwritten Signature Modeling and Verification Based on Archetypal Analysis , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[9]  Marcus Liwicki,et al.  Offline Signature Verification by Combining Graph Edit Distance and Triplet Networks , 2018, S+SSPR.

[10]  Lianwen Jin,et al.  Learning Discriminative Feature Hierarchies for Off-Line Signature Verification , 2018, 2018 16th International Conference on Frontiers in Handwriting Recognition (ICFHR).

[11]  Kaiqi Huang,et al.  Beyond Triplet Loss: A Deep Quadruplet Network for Person Re-identification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  George Economou,et al.  Offline signature verification and quality characterization using poset-oriented grid features , 2016, Pattern Recognit..

[14]  Luiz Eduardo Soares de Oliveira,et al.  Learning features for offline handwritten signature verification using deep convolutional neural networks , 2017, Pattern Recognit..

[15]  Lucas Beyer,et al.  In Defense of the Triplet Loss for Person Re-Identification , 2017, ArXiv.

[16]  Stefanos Zafeiriou,et al.  ArcFace: Additive Angular Margin Loss for Deep Face Recognition , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Babak Nadjar Araabi,et al.  Deep Multitask Metric Learning for Offline Signature Verification , 2016, Pattern Recognit. Lett..

[18]  Sargur N. Srihari,et al.  Offline Signature Verification And Identification Using Distance Statistics , 2004, Int. J. Pattern Recognit. Artif. Intell..

[19]  Yann LeCun,et al.  Dimensionality Reduction by Learning an Invariant Mapping , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[20]  Errui Ding,et al.  Multi-Attention Multi-Class Constraint for Fine-grained Image Recognition , 2018, ECCV.

[21]  Kihyuk Sohn,et al.  Improved Deep Metric Learning with Multi-class N-pair Loss Objective , 2016, NIPS.

[22]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Yann LeCun,et al.  Signature Verification Using A "Siamese" Time Delay Neural Network , 1993, Int. J. Pattern Recognit. Artif. Intell..

[24]  Richard S. Zemel,et al.  Prototypical Networks for Few-shot Learning , 2017, NIPS.

[25]  Matthew R. Scott,et al.  Multi-Similarity Loss With General Pair Weighting for Deep Metric Learning , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Bhiksha Raj,et al.  SphereFace: Deep Hypersphere Embedding for Face Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Christoph Meinel,et al.  Signature Embedding: Writer Independent Offline Signature Verification with Deep Metric Learning , 2016, ISVC.

[28]  Li Liu,et al.  Off-Line Signature Verification Using a Region Based Metric Learning Network , 2018, PRCV.

[29]  Marcus Liwicki,et al.  Signature Verification Competition for Online and Offline Skilled Forgeries (SigComp2011) , 2011, 2011 International Conference on Document Analysis and Recognition.

[30]  Nir Ailon,et al.  Deep Metric Learning Using Triplet Network , 2014, SIMBAD.

[31]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).