Detecting central units in argumentative answer genre : signals that influence annotators ' agreement

This paper aims at investigating annotators' agreement in central unit detection in a corpus of one hundred texts of argumentative answer genre. The theory underlying the investigation is Rhetorical Structure Theory (RST), proposed by Mann and Thompson (1988), which aims at investigating text coherence regarding relations held between parts of text, both in macro and microstructure. The research unveiled the most common pattern of central units: the central unit is the initial statement of the text; the answer resumes the question and the central unit presents the structure “The secret of Vestibular + copula verb + factor(s)” or a structure which takes into account more than one factor; the initial statement may be embedded in evidential verbs of propositional attitude and adverbs may also be used, specially epistemic asseverative adverbs. Discrepancies were motivated by annotators' mistakes or by writers' mistakes. Annotators' mistakes were the following: identifying the conclusion as the central unit because the conclusion seems more complete and stronger than the initial statement; considering some word in the text a stronger indicator of the central unit than the pattern presented previously. Writers' mistakes were the following: choosing one factor and further up the text choosing another factor(s) as the secret of Vestibular; conveying the same answer twice by paraphrasing. The results of this paper may be useful for the development of automatic evaluation applications.