Applying a Convolutional Neural Network to Legal Question Answering

Our legal question answering system combines legal information retrieval and textual entailment, and we describe a legal question answering system that exploits a deep convolutional neural network. We have evaluated our system using the training/test data from the competition on legal information extraction/entailment (COLIEE). The competition focuses on the legal information processing related to answering yes/no questions from Japanese legal bar exams, and it consists of three phases: ad-hoc legal information retrieval, textual entailment, and a learning model-driven combination of the two phases. Phase 1 requires the identification of Japan civil law articles relevant to a legal bar exam query. For that phase, we have implemented a combined TF-IDF and Ranking SVM information retrieval component. Phase 2 requires the system to answer “Yes” or “No” to previously unseen queries, by comparing extracted meanings of queries with relevant articles. Our training of an entailment model focuses on features based on word embeddings, syntactic similarities and identification of negation/antonym relations. We augment our textual entailment component with a convolutional neural network with dropout regularization and Rectified Linear Units. To our knowledge, our study is the first to adapt deep learning for textual entailment. Experimental evaluation demonstrates the effectiveness of the convolutional neural network and dropout regularization. The results show that our deep learning-based method outperforms our baseline SVM-based supervised model and K-means clustering.