Multitask learning approach for understanding the relationship between two sentences

Abstract Understanding the relationship between two sentences through tasks including natural language inference, paraphrase detection, semantic textual similarity, and semantic relatedness is a fundamental step to natural language understanding. We propose an approach to infer the relationship between two sentences using a multitask framework to generate a universal representation of the relationship. Our model consists of a universal layer shared for all tasks with several task-specific layers on top for each task. To generate universal representation, we employ the enhanced sequential inference model based on a deep learning and soft alignment techniques. The task-specific layers are composed of multilayer perceptrons. The main feature of the proposed approach is that a single encoder can model various relationship of sentences at same time on multiple tasks. When we evaluated our approach on four public datasets for four different tasks regarding the relationship between two sentences, it outperformed state-of-the-art methods for two datasets and performed significantly well for the other two datasets. Further investigation of our proposed model showed that it captures comprehensive information together with specific knowledge regarding each task to infer semantic similarity. The detailed analysis supports that the proposed approach is robust over all semantic inference tasks using a single model.