VENSES - A Linguistically-Based System for Semantic Evaluation

The system for semantic evaluation VENSES (Venice Semantic Evaluation System) is organized as a pipeline of two subsystems: the first is a reduced version of GETARUN, our system for Text Understanding. The output of the system is a flat list of head-dependent structures (HDS) with Grammatical Relations (GRs) and Semantic Roles (SRs) labels. The evaluation system is made up of two main modules: the first is a sequence of linguistic rule-based subcalls; the second is a quantitatively based measurement of input structures. VENSES measures semantic similarity which may range from identical linguistic items, to synonymous or just morphologically derivable. Both modules go through General Consistency checks which are targeted to high level semantic attributes like presence of modality, negation, and opacity operators, temporal and spatial location checks. Results in cws, accuracy and precision are homogenoues for both training and test corpus and fare higher than 60%

[1]  Rodolfo Delmonte,et al.  Binding Pronominals with an LFG Parser , 1991, IWPT.

[2]  Christopher D. Manning,et al.  Robust Textual Inference using Diverse Knowledge Sources , 2005 .

[3]  Sanda M. Harabagiu,et al.  Open-domain textual question answering techniques , 2003, Natural Language Engineering.

[4]  Rodolfo Delmonte,et al.  GETARUN PARSER - A parser equipped with Quantifier Raising and Anaphoric Binding based on LFG , 2002 .

[5]  Rodolfo Delmonte.  Evaluating GETARUNS parser with GREVAL test suite , 2004, COLING 2004.

[6]  van Gerardus Noord,et al.  Special issue: finite state methods in natural language processing , 2003 .

[7]  Rodolfo Delmonte,et al.  Semantic parsing with LFG and conceptual representations , 1990, Comput. Humanit..

[8]  Ido Dagan,et al.  The Third PASCAL Recognizing Textual Entailment Challenge , 2007, ACL-PASCAL@ACL.

[9]  Dan Roth,et al.  Natural Language Inference via Dependency Tree Mapping: An Application to Question Answering , 2004 .

[10]  Chris Quirk,et al.  Unsupervised Construction of Large Paraphrase Corpora: Exploiting Massively Parallel News Sources , 2004, COLING.

[11]  Patrick Pantel,et al.  DIRT @SBT@discovery of inference rules from text , 2001, KDD '01.

[12]  Ido Dagan,et al.  A Probabilistic Classification Approach for Lexical Textual Entailment , 2005, AAAI.

[13]  Dan Roth,et al.  An Inference Model for Semantic Entailment in Natural Language , 2005, IJCAI.

[14]  Rodolfo Delmonte,et al.  Getaruns: a hybrid system for summarization and question answering , 2003 .

[15]  Lauri Karttunen,et al.  Local Textual Inference: Can it be Defined or Circumscribed? , 2005, EMSEE@ACL.

[16]  Mark Steedman,et al.  Wide-Coverage Semantic Representations from a CCG Parser , 2004, COLING.

[17]  Ca'Garzoni-Moro.  GETARUN PARSER A parser equipped with Quantifier Raising and Anaphoric Binding based on LFG , 2002 .

[18]  Rodolfo Delmonte.  Text Understanding with GETARUNS for Q/A and Summarization , 2004, ACL 2004.

[19]  Rada Mihalcea,et al.  Measuring the Semantic Similarity of Texts , 2005, EMSEE@ACL.