University of Pittsburgh

Narrowing the gap in AI & Legal Argumentation: Case Outcome Prediction and Semantic Analysis of Court Opinions

Friday, September 9, 2016 - 12:30pm - 1:30pm

Artificial Intelligence and Law studies how legal reasoning can be formalized in order to eventually be able to develop systems that assist lawyers in the task of researching, drafting and evaluating arguments in a professional setting. To further this goal, researchers have been developing systems, which, to a limited extent, autonomously engage in legal reasoning, and argumentation on closed domains. However, the population of such systems with formalized domain knowledge is the the main bottleneck preventing such systems from making real practical contributions. This talk discusses prospects to narrow this gap by presenting (1) recent dissertation work at Pitt ISP on top-down conceptual refinement of deep legal argumentation and case outcome prediction, as well as (2) a collaborative project by Pitt LRDC, CMU LTI and Hofstra Law School on bottom-up semantic analysis of legal documents.

The first part presents the Value Judgment Formalism and its experimental implementation in the VJAP system, which is capable of arguing about, and predicting outcomes of, a set of trade secret misappropriation cases. VJAP argues about cases by creating an argument graph for each case using a set of argument schemes. These schemes use a representation of values underlying trade secret law and effects of facts on these values. VJAP argumentatively balances effects in the given case and analogizes it to individual precedents and the value tradeoffs in those precedents. It predicts case outcomes using a confidence measure computed from the argument graph and generates textual legal arguments justifying its predictions. The confidence propagation uses quantitative weights assigned to effects of facts on values. VJAP automatically learns these weights from past cases using an iterative optimization method. 

The second part presents results from ongoing work in conceptual legal document retrieval in a particular domain involving vaccine injury compensation. The conceptual markup of documents is done automatically using LUIMA, a law-specific semantic extraction toolbox based on the UIMA framework. The system consists of modules for automatic sub-sentence level annotation, machine learning based sentence annotation, basic retrieval using Apache Lucene and a machine learning based reranking of retrieved documents. Experiments on a limited corpus show that the resulting rankings scored higher for most tested queries than baseline rankings created using a commercial full-text legal information system. Error analysis and further work has shown that performance can be improved by feature engineering around characteristic phenomena in legal opinions such as citations to prior cases.

Copyright 2009–2020 | Send feedback about this site