论文信息 - Passing the Brazilian OAB Exam: Data Preparation and Some Experiments

Passing the Brazilian OAB Exam: Data Preparation and Some Experiments

In Brazil, all legal professionals must demonstrate their knowledge of the law and its application by passing the OAB exams, the national bar exams. The OAB exams therefore provide an excellent benchmark for the performance of legal information systems since passing the exam would arguably signal that the system has acquired capacity of legal reasoning comparable to that of a human lawyer. This article describes the construction of a new data set and some preliminary experiments on it, treating the problem of finding the justification for the answers to questions. The results provide a baseline performance measure against which to evaluate future improvements. We discuss the reasons to the poor performance and propose next steps.

[1] Alexander F. Gelbukh,et al. Using Graphs for Shallow Question Answering on Legal Documents , 2008, MICAI.

[2] Jeff Z. Pan,et al. Passing a USA National Bar Exam: a First Corpus for Experimentation , 2016, LREC.

[3] Diego Calvanese,et al. The Description Logic Handbook: Theory, Implementation, and Applications , 2003, Description Logic Handbook.

[4] Valeria de Paiva,et al. Intuitionistic Logic and Legal Ontologies , 2010, JURIX.

[5] Hinrich Schütze,et al. Introduction to information retrieval , 2008 .

[6] Hans Kelsen,et al. General Theory of Norms , 1991 .

[7] Ruslan Mitkov,et al. The Oxford handbook of computational linguistics , 2003 .

[8] Alexander F. Gelbukh,et al. NLP for Shallow Question Answering of Legal Documents Using Graphs , 2009, CICLing.

[9] Gerard de Melo,et al. OpenWordNet-PT: An Open Brazilian Wordnet for Reasoning , 2012, COLING.

[10] Valeria de Paiva,et al. Constructive Description Logics Hybrid-Style , 2011, Electron. Notes Theor. Comput. Sci..

[11] Valeria C V de Paiva,et al. Using Intuitionistic Logic as a Basis for Legal Ontologies , 2010 .

[12] Paulo Quaresma,et al. A question-answering system for Portuguese juridical documents , 2005, ICAIL '05.

[13] Alexandre Rademaker,et al. An Intuitionisticaly based Description Logic , 2014, ArXiv.