Evaluating the Premises and Results of Four Metaphor Identification Systems

This study first examines the implicit and explicit premises of four systems for identifying metaphoric utterances from unannotated input text. All four systems are then evaluated on a common data set in order to see which premises are most successful. The goal is to see if these systems can find metaphors in a corpus that is mostly non-metaphoric without over-identifying literal and humorous utterances as metaphors. Three of the systems are distributional semantic systems, including a source-target mapping method [1-4]; a word abstractness measurement method [5], [6, 7]; and a semantic similarity measurement method [8, 9]. The fourth is a knowledge-based system which uses a domain interaction method based on the SUMO ontology [10, 11], implementing the hypothesis that metaphor is a product of the interactions among all of the concepts represented in an utterance [12, 13].