TVAE : Deep Metric Learning Approach for Variational Autoencoder

Deep metric learning has been demonstrated to be highly effective in learning semantic representation and encoding data information. People are able to do similarity measurement for data, based on the embedding learned from metric learning. At the same time, variational autoencoder (VAE) has widely been used to approximate inference and proved to have a good performance for directed probabilistic models. However, for traditional VAE, the data label or feature information are intractable. Similarly, traditional representation learning approaches fail to represent many salient aspects of the data. To this end, in this project, we propose a novel structure to learn latent embedding in VAE by incorporating deep metric learning. The features are learned by a triplet loss on the mean vectors of VAE in conjunction with reconstruction loss of VAE. This approach, which we call Triplet based Variational Autoencoder (TVAE), allows us to capture more fine-grained information in the embedding. Our model is first tested on MNIST data set. A high triplet accuracy of around 95.60% is achieved while the VAE is found to perform well at the same time. We further implement our structure on Zappos50k shoe dataset [32] to show the efficacy of our method.