Siamese Convolutional Neural Networks for Authorship Verification

Determining handwriting authorship of a written text has practical significance in the realm of forensics, signature verification, and literary history. While there have been studies in signature verification and handwriting classification, a vast literature review reveals that very little work has been done in handwriting verification. Recent advances in convolutional architectures, particularly those involving facial verification, suggest that the task can be tackled effectively. In this study, we build a Siamese convolutional neural network to determine whether two pieces of handwriting are written by the same author. We examine questions such as whether long pieces of handwriting must be present to achieve good results, how many samples are needed, what features are important, and how different architectures perform on this task. We explore different convolutional architectures like VGG, GoogLeNet and ResNet, to determine which architecture produces the best encoding of each sample. We note that our best performing single model, TinyResNet, achieves a 92.08% accuracy on the held out test set.