Automatic Extraction of Subcategorization Frames from Corpora -improving Filtering with Diathesis Alternations

Attempts to extract subcategorization information from textual corpora by shallow parsing followed by statistical ltering of alternatives proposed for speciic predicates have met with some success (Briscoe & Carroll, 1997) but are not yet accurate enough. Examination of the errors suggests that the ltering of spurious hypotheses is the source of most errors in the system. This paper builds on the framework described in (Briscoe and Carroll, 1997) and proposes a knowledge-based approach for improvement of the ltering phase of the system. 1 Background Manual development of large subcategorised lexicons has proved very diicult because predicates change behaviour between sublanguages, domains and across time. Yet current parsers depend crucially on such information, and probabilistic parsers would greatly beneet from accurate information concerning relative likelihood of diierent subcategorization frames of a given predicate. This suggests that automatic construction of subcategorization dictionaries from textual corpora is a more promising method to apply. Briscoe & Carroll (1997) propose a technique and implemented system for constructing a subcategorization dictionary from textual corpora. Their system is capable of distinguishing 160 subcategorization classes, and able to both assign classes to individual verbal predicates and to rank them according to relative frequency. As described in Briscoe & Carroll (1997), the system consists of six overall components which are applied in sequence to sentences containing a speciic predicate in order to retrieve a set of subcategoriza-tion classes for that predicate: a tagger, a lemmatizer, a probabilistic LR parser, a patternset extractor, a pattern classiier, and a patternsets evaluator. Even though the system has met with some success it is not yet accurate enough. The experimental evaluation performed by Briscoe & Carroll shows that the ltering of spurious hypotheses in the patternsets evaluator stage is the weak link of the system. 2 Filtering In the current lter, the set of putative classes are ltered, following Brent (1993) by hypothesis testing on binomial frequency data. The system rst records the total number of patternsets n for a given predicate, the number of these patternsets containing a pattern supporting an entry for given class m, and estimates of the probability that a pattern for a class i will occur with a verb which is not a member of subcategorization class i. Briscoe & Carroll estimate the above probability by rst extracting the number of verbs which are members of each class in the ANLT dictionary (Boguraev et al. 1987), and converting …