Deep Neural Networks with Parallel Autoencoders for Learning Pairwise Relations: Handwritten Digits Subtraction

Modelling relational data is a common task for many machine learning problems. In this work, we focus on learning pairwise relations between two entities, with deep neural networks. To incorporate the structural properties in the data that represent two entities concatenated together, two separate stacked autoencoders are introduced in parallel to extract individual features, which are then fed into a deep neural network for classification. The method is applied to a specific problem: whether two input handwritten digits differ by one. We tested the performance on a dataset generated from handwritten digits in MNIST, which is a widely used dataset for testing different machine learning techniques and pattern recognition methods. We compared with several different machine learning algorithms, including logistic regression and support vector machines, on this handwritten digit subtraction (HDS) dataset. The results showed that deep neural networks outperformed other methods, and in particular, the deep neural networks fitted with two separate autoencoders in parallel increased the prediction accuracy from 85.83%, which was achieved by a standard neural network with a single stacked autoencoder, to 88.27%.