What matters in a transferable neural network model for relation classification in the biomedical domain?

A lack of sufficient labeled data often limits the applicability of advanced machine learning algorithms to real life problems. However, the efficient use of transfer learning (TL) has been shown to be very useful across domains. TL make use of valuable knowledge learned in one task (source task), where sufficient data is available, in order to improve performance on the task of interest (target task). In the biomedical and clinical domain, a lack of sufficient training data means that machine learning models cannot be fully exploited. In this work, we present two unified recurrent neural models leading to three transfer learning frameworks for relation classification tasks. We systematically investigate the effectiveness of the proposed frameworks in transferring knowledge from a source task to a target task when the characteristics of the source data vary, such as similarity or relatedness between the source and target tasks, and the size of training data for the source task. Our empirical results show that the proposed frameworks, in general, improve the model performance. However, these improvements do depend on characteristics of source and target tasks. This dependence then finally determine the choice of a particular TL framework.