Automatic Noun Compound Interpretation using Deep Neural Networks and Word Embeddings

The present paper reports on the results of automatic noun compound interpretation for English using a deep neural network classifier and a selection of publicly available word embeddings to represent the individual compound constituents. The task at hand consists of identifying the semantic relation that holds between the constituents of a compound (e.g. WHOLE+PART_OR_MEMBER_OF in the case of ‘robot arm’, LOCATION in the case of ‘hillside home’). The experiments reported in the present paper use the noun compound dataset described in Tratz (2011), a revised version of the dataset used by Tratz and Hovy (2010) for training their Maximum Entropy classifier. Our experiments yield results that are comparable to those reported in Tratz and Hovy (2010) in a crossvalidation setting, but outperform their system on unseen compounds by a large margin.

[1]  Diarmuid Ó Séaghdha Learning compound noun semantics , 2008 .

[2]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[3]  Preslav Nakov,et al.  SemEval-2013 Task 4: Free Paraphrases of Noun Compounds , 2013, SemEval@NAACL-HLT.

[4]  Barbara Rosario,et al.  Classifying the Semantic Relations in Noun Compounds via a Domain-Specific Lexical Hierarchy , 2001, EMNLP.

[5]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[6]  Léon Bottou,et al.  Stochastic Gradient Descent Tricks , 2012, Neural Networks: Tricks of the Trade.

[7]  D I A R M U I D ´ O S ´ E A G H D H A And A N N C O Interpreting compound nouns with kernel methods , 2012 .

[8]  Jason Weston,et al.  A unified architecture for natural language processing: deep neural networks with multitask learning , 2008, ICML '08.

[9]  Stephen Tratz Semantically-enriched parsing for natural language understanding , 2011 .

[10]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[11]  Clément Farabet,et al.  Torch7: A Matlab-like Environment for Machine Learning , 2011, NIPS 2011.

[12]  Walter Daelemans,et al.  Classification of noun-noun compound semantics in Dutch and Afrikaans , 2012 .

[13]  Dan I. Moldovan,et al.  On the semantics of noun compounds , 2005, Comput. Speech Lang..

[14]  Timothy Baldwin,et al.  Automatic Interpretation of Noun Compounds Using WordNet Similarity , 2005, IJCNLP.

[15]  Stan Szpakowicz,et al.  Semi-Automatic Recognition of Noun Modifier Relationships , 1998, ACL.

[16]  Lutz Prechelt,et al.  Early Stopping - But When? , 2012, Neural Networks: Tricks of the Trade.

[17]  Eduard H. Hovy,et al.  A Taxonomy, Dataset, and Classifier for Automatic Noun Compound Interpretation , 2010, ACL.

[18]  Ronan Collobert,et al.  Word Embeddings through Hellinger PCA , 2013, EACL.