Learning to Rank Sentences for Explaining Statutory Terms

We explore using classical feature engineering-based learning-to-rank approaches (LTR) to discover sentences for explaining the meaning of statutory terms. We compiled a list of 129 descriptive features that model retrieved sentences, their relationships to statutory terms, and their statutory provisions of origin. Using a statutory interpretation data set (26,959 sentences) we showed how the proposed feature set could be utilized in learning-to-rank settings with reasonable effectiveness. We showed that off-the-shelf machine learning algorithms perform significantly better than BM25 baselines (NDCG@100 of 0.77 vs 0.68).

[1]  Chen Wang,et al.  Introducing LUIMA: an experiment in legal conceptual retrieval of vaccine injury decisions using a UIMA type system and tools , 2015, ICAIL.

[2]  Nada Lavrac,et al.  LemmaGen: Multilingual Lemmatisation with Induced Ripple-Down Rules , 2010, J. Univers. Comput. Sci..

[3]  Vern R. Walker.  Argumentation Mining from Judicial Decisions : The Attribution Problem and the Need for Legal Discourse Models , 2015 .

[4]  S. Holm.  A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[5]  Kevin D. Ashley,et al.  From Information Retrieval (IR) to Argument Retrieval (AR) for Legal Cases: Report on a Baseline Study , 2013, JURIX.

[6]  Erich Schweighofer,et al.  Legal Query Expansion using Ontologies and Relevance Feedback , 2007, LOAIT.

[7]  Kevin D. Ashley,et al.  Segmenting U.S. Court Decisions into Functional and Issue Specific Parts , 2018, JURIX.

[8]  Harris Drucker,et al.  Improving Regressors using Boosting Techniques , 1997, ICML.

[9]  Vern R. Walker,et al.  Semantic types for computational legal reasoning: propositional connectives and sentence roles in the veterans' claims dataset , 2017, ICAIL.

[10]  Jakub Harasta,et al.  Open Texture in Law, Legal Certainty and Logical Analysis of Natural Language , 2015, Logic in the Theory and Practice of Lawmaking.

[11]  Günther Grewendorf,et al.  Definition extraction from court decisions using computational linguistic technology , 2009 .

[12]  Eibe Frank,et al.  A Simple Approach to Ordinal Classification , 2001, ECML.

[13]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[14]  Kevin D. Ashley,et al.  Document Ranking with Citation Information and Oversampling Sentence Classification in the LUIMA Framework , 2016, JURIX.

[15]  Tao Qin,et al.  LETOR: A benchmark collection for research on learning to rank for information retrieval , 2010, Information Retrieval.

[16]  Raimo Siltala,et al.  Interpreting Statutes. A Comparative Study , 1993 .

[17]  Balaji Padmanabhan,et al.  A process model for information retrieval context learning and knowledge discovery , 2015, Artificial Intelligence and Law.

[18]  M. Saravanan,et al.  Improving legal information retrieval using an ontological framework , 2009, Artificial Intelligence and Law.

[19]  Kevin D. Ashley,et al.  Improving Sentence Retrieval from Case Law for Statutory Interpretation , 2019, ICAIL.

[20]  Kevin D. Ashley,et al.  Sentence Boundary Detection in Adjudicatory Decisions in the United States , 2017, TAL.

[21]  Tie-Yan Liu,et al.  Learning to rank for information retrieval , 2009, SIGIR.

[22]  Kevin D. Ashley,et al.  Extracting Case Law Sentences for Argumentation about the Meaning of Statutory Terms , 2016, ArgMining@ACL.

[23]  M. Friedman.  The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance , 1937 .