Data analysis of conceptual similarities of Finnish verbs

Data analysis of conceptual similarities of Finnish verbs Krista Lagus (krista.lagus@hut.fi) Neural Networks Research Centre, Helsinki University of Technology P.O.Box 9800, 02015 HUT, Finland Anu Airola (anu.airola@helsinki.fi) Department of General Linguistics, University of Helsinki P.O.Box 9, 00014 University of Helsinki, Finland Mathias Creutz (mathias.creutz@hut.fi) Neural Networks Research Centre, Helsinki University of Technology P.O.Box 9800, 02015 HUT, Finland Abstract The study of the conceptual representations that underlie the use of language is a problem motivated from both a cognitive research point of view and that of construing language models for various lan- guage processing tasks. In this work, we organized 600 Finnish verbs using the SOM algorithm. Three experiments were conducted using different features to encode the verbs: morphosyntactic properties, individual nouns, and noun categories in the con- text of the verb. In general, the morphosyntactic properties seem to draw attention to semantic roles, whereas nouns as features seem to highlight clusters formed on grounds of topics in the text. Introduction Observation of language use provides indirect ev- idence of the representations that humans utilize. The study of conceptual representations that un- derlie the use of language is important for applica- tions such as speech recognition. Due to the redun- dancy in communication, by studying large amounts of data it may be possible to induce the concep- tual, system-internal representations which provide a grounding for meanings of words. Whether this is possible, and if so, how, is an interesting and con- troversial question. A central problem in learning a language or in estimating a language model-1 from data is how to generalize from particular observations to new, sim- ilar instances. Generalization requires knowledge of similarities between words, concepts and other units of language and thought, i.e., similarity representa- tions. The hypothesis that the semantic similarity of two words correlates strongly with the similarity of their contexts has been widely discussed in linguistics and psychology (for recent treatments, see Levin, 1993 and Miller & Charles, 1991). It has been proposed by Gardenfors that a cen- tral part of our conceptual reprensentations are 1For an introduction to statistical language modeling see (Manning & Schiitze, 1999). Their applications in- clude speech recognition, machine translation, and di- alogue agents that converse with humans in order to perform tasks such as answering questions about train schedules and booking flights. grounded in various low-dimensional conceptual spaces. A conceptual space is defined as a set of qual- ity dimensions with a geometrical structure (Gar- denfors, 2000). Examples of conceptual spaces near our perceptual apparatus are colors and the pitch of sounds. For many higher order concepts a geo- metric interpretation can be found, as well. For ex- ample, comparative relations such as ‘longer than’ can be represented as a geometric relation between two elementary length spaces. Gardenfors proposes a subset of concepts called natural concepts: A natural concept is represented as a set of re- gions in a number of domains together with an assignment of salience weights to the domains and information about how the regions in dif- ferent domains are correlated. An inherent and important property of the pro- posed conceptual spaces is that they provide a mean- ing representation that is ordered and offers means for representing similarities, often in terms of some continuous-valued underlying qualities. Gardenfors gives examples of conceptual spaces that humans are likely to have. However, an open research question remains for both brain research and the study of language use: What are the possible conceptual di- mensions that humans utilize‘? In this work we analyze the use of Finnish2 verbs with the following goals in mind: (a) to uncover pos- sible conceptual spaces, i.e., underlying, organizing semantic qualities or properties, (b) to study seman- tic similarities of verbs in actual language use. In particular, we examine the kinds of semantic or con- ceptual ordering qualities that appear to affect the distribution of features in the immediate context of a verb, in particular (1) morphosyntactic properties of nearby words, and (2) the nearby nouns and (3) unsupervised categories of nearby nouns. In effect, we rely on the redundancy in communication and assume that certain regularities observed in the dis- tributions of verb contexts will contain significant information about the semantics of the verb as well. 2Most of the research on language is carried out us- ing English data only, which creates a too narrow or misleading picture of the modeling apparatus underly- ing language learning and use.