A novel Enhanced Collaborative Autoencoder with knowledge distillation for top-N recommender systems

Abstract In most recommender systems, the data of user feedbacks are usually represented with a set of discrete values, which are difficult to exactly describe users’ interests. This problem makes it not easy to exactly model users’ latent preferences for recommendation. Intuitively, a basic idea for this issue is to predict continuous values through a trained model to reveal users’ essential feedbacks, and then make use of the generated data to retrain another model to learn users’ preferences. However, since these continuous data are generated by an imperfect model which are trained by discrete data, there exists a lot of noise among the generated data. This problem may have a severe adverse impact on the performance. Towards this problem, we propose a novel Enhanced Collaborative Autoencoder (ECAE) to learn robust information from generated soft data with the technique of knowledge distillation. First, we propose a tightly coupled structure to incorporate the generation and retraining stages into a unified framework. So that the generated data can be fine tuned to reduce the noise by propagating training errors of retraining network. Second, for that each unit of the generated data contains different level of noise, we propose a novel distillation layer to balance the influence of noise and knowledge. Finally, we propose to take both predict results of generation and retraining network into account to make final recommendations for each user. The experimental results on four public datasets for top-N recommendation show that the ECAE model performs better than several state-of-the-art algorithms on metrics of MAP and NDCG.