Idioms: Humans or Machines, It's All About Context

Expressions can be ambiguous between idiomatic and literal interpretation depending on the context they occur in (“sales hit the roof” vs “hit the roof of the car”). Previous studies suggest that idiomaticity is not a binary property, but rather a continuum or the so-called “scalar phenomenon” ranging from completely literal to highly idiomatic. This paper reports the results of an experiment in which human annotators rank idiomatic expressions in context on a scale from 1 (literal) to 4 (highly idiomatic). Our experiment supports the hypothesis that idioms fall on a continuum and that one might differentiate between highly idiomatic, mildly idiomatic and weakly idiomatic expressions. In addition, we measure the relative idiomaticity of 11 idiomatic types and compute the correlation between the relative idiomaticity of an expression and the performance of various automatic models for idiom detection. We show that our model, based on the distributional semantics ideas, not only outperforms the previous models, but also positively correlates with the human judgements, which suggests that we are moving in the right direction toward automatic idiom detection.

[1]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[2]  Aravind K. Joshi,et al.  Measuring the Relative Compositionality of Verb-Noun (V-N) Collocations by Integrating Features , 2005, HLT.

[3]  Caroline Sporleder,et al.  Unsupervised Recognition of Literal and Non-Literal Use of Idiomatic Expressions , 2009, EACL.

[4]  Afsaneh Fazly,et al.  Pulling their Weight: Exploiting Syntactic Forms for the Automatic Identification of Idiomatic Expressions in Context , 2007 .

[5]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[6]  Carlos Ramisch,et al.  Predicting the Compositionality of Nominal Compounds: Giving Word Embeddings a Hard Time , 2016, ACL.

[7]  Aline Villavicencio,et al.  Lexical Encoding of MWEs , 2004 .

[8]  Jing Peng,et al.  Experiments in Idiom Recognition , 2016, COLING.

[9]  Suzanne Stevenson,et al.  The VNC-Tokens Dataset , 2008 .

[10]  Timothy Baldwin,et al.  A Word Embedding Approach to Predicting the Compositionality of Multiword Expressions , 2015, NAACL.

[11]  A. Feinstein,et al.  High agreement but low kappa: I. The problems of two paradoxes. , 1990, Journal of clinical epidemiology.

[12]  Ekaterina Vylomova,et al.  Classifying Idiomatic and Literal Expressions Using Topic Models and Intensity of Emotions , 2014, EMNLP.

[13]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[14]  Cristina Cacciari,et al.  The place of idioms in a literal and metaphorical world. , 1993 .

[15]  Timothy Baldwin,et al.  Multiword Expressions: A Pain in the Neck for NLP , 2002, CICLing.

[16]  Marti A. Hearst.  Automatic Acquisition of Hyponyms from Large Text Corpora , 1992, COLING.

[17]  Stefanie Wulff,et al.  Rethinking Idiomaticity: A Usage-based Approach , 2009 .

[18]  John Carroll,et al.  Detecting a Continuum of Compositionality in Phrasal Verbs , 2003, ACL 2003.

[19]  Meghdad Farahmand,et al.  Learning Semantic Composition to Detect Non-compositionality of Multiword Expressions , 2015, EMNLP.

[20]  John D. Kelleher,et al.  Idiom Token Classification using Sentential Distributed Semantics , 2016, ACL.

[21]  Afsaneh Fazly,et al.  Unsupervised Type and Token Identification of Idiomatic Expressions , 2009, CL.

[22]  Hadas Kotek,et al.  A streamlined approach to online linguistic surveys , 2016 .

[23]  Suresh Manandhar,et al.  An Empirical Study on Compositionality in Compound Nouns , 2011, IJCNLP.

[24]  Eugenie Giesbrecht,et al.  Automatic Identification of Non-Compositional Multi-Word Expressions using Latent Semantic Analysis , 2006 .

[25]  Paul M. B. Vitányi,et al.  Normalized Web Distance and Word Similarity , 2009, Handbook of Natural Language Processing.

[26]  J. R. Firth,et al.  A Synopsis of Linguistic Theory, 1930-1955 , 1957 .

[27]  Anoop Sarkar,et al.  A Clustering Approach for Nearly Unsupervised Recognition of Nonliteral Language , 2006, EACL.

[28]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[29]  Joakim Nivre,et al.  A Multiword Expression Data Set: Annotating Non-Compositionality and Conventionalization for English Noun Compounds , 2015, MWE@NAACL-HLT.

[30]  Jing Peng,et al.  Classifying Idiomatic and Literal Expressions Using Vector Space Representations , 2015, RANLP.

[31]  Caroline Sporleder,et al.  Using Gaussian Mixture Models to Detect Figurative Language in Context , 2010, NAACL.

[32]  I. Sag,et al.  Idioms , 2015 .

[33]  Ari Rappoport,et al.  Multi-Word Expression Identification Using Sentence Surface Features , 2009, EMNLP.

[34]  Xiaoyan Zhu,et al.  Measuring the Non-compositionality of Multiword Expressions , 2010, COLING.

[35]  Paul M. B. Vitányi,et al.  The Google Similarity Distance , 2004, IEEE Transactions on Knowledge and Data Engineering.

[36]  Christiane Fellbaum,et al.  Corpus-based Studies of German Idioms and Light Verbs , 2006 .

[37]  Dominic Widdows,et al.  Automatic Extraction of Idioms using Graph Analysis and Asymmetric Lexicosyntactic Patterns , 2005, ACL 2005.

[38]  Shrikant I Bangdiwala,et al.  Observer agreement paradoxes in 2x2 tables: comparison of agreement measures , 2014, BMC Medical Research Methodology.

[39]  S. Glucksberg.  Understanding figurative language : from metaphors to idioms , 2001 .