Lexical Attraction Models of Language

Abstract ID: A229This paper presents lexical attraction models of lan-guage, in which the only explicitly represented linguis-tic knowledge is the likelihood of pairwise relations be-tween words. This is in contrast with models that rep-resent linguistic knowledge in terms of a lexicon, whichassigns categories to each word, and a grammar, whichexpresses possible combinations in terms of these cat-egories. The word-based nature and the simplicity oflexical attraction models make them good candidatesfor experiments in language learning. I introduce anunsupervised learning algorithm that uses lexical at-traction and gives accuracy results comparable to su-pervised learning. Content Areas: Natural Language Processing #Techniques or Algorithms # statistical or corpus basedmethods, Machine Learning and Discovery # Tasks orProblems # unsupervised learning Introduction The information in a sentence is contained partly inits words and partly in the relationships between thewords. The main task of natural language processing isto identify the relationships between the words. Whendeciding whether two words are related we typically usetwo types of information. First, there are grammaticalconstraints, e.g. each determiner must modify a nounor transitive verbs must take objects. However, gram-matical constraints are typically not restrictive enoughto uniquely identify the correct relations. Second, thereare selectional restrictions, e.g. given the verb eat, cakewouldbe a morelikely objectthan train. Lexicalattrac-tion models encode selectional restrictions. Ordinarily,one would need both types of information to identifythe correct structure of a sentence. I will show how farwe can go using lexical attraction alone.The next section presents examples that demonstratewhen grammatical constraints are not sufficient andlexical attraction is. The third section formalizes themodel and expresses the meaning of my opening sen-tence in the language of information theory. The nextsection gives some basic results on dependency struc-tures. The section on unsupervised learning gives thelearning algorithm and evaluation results. The pa-per ends with a discussion of related work and futureprospects.