Treebanking in Northern Europe: A White Paper

We present the case for an extensive scientific effort to build up large treebanks for the Nordic and Baltic languages, as a step towards developing advanced multilingual communication technologies for these languages in the future. Nordic language technology is urgent Language and speech processing is rapidly becoming a priority area for Northern Europe (in which we include the Baltic states). Two recent, broad developments are causes of this urgency. First, the speed at which digital information technologies have penetrated our society has accelerated particularly strongly in Northern Europe. Secondly, the expansion of the European Union has had an impact on the interaction between these countries, in particular between the Nordic states and the Baltic states. The challenges for multilingual text and speech processing are enormous. The Northern European area comprises about 31.5 million people (not including Northern Russia) speaking 8 official languages (9 including Russian) and several minority languages. Roughly 45% of all EU inhabitants are able to converse in at least two different languages. The language that is most often the common denominator in Europe, also in Northern Europe, is English, although in the Baltic states and Russia, English language competency is lower, for historical reasons. From a political viewpoint, it is not acceptable to promote English at the expense of the languages of the region. Most of the information on the Internet consists of natural language, of which less than 50% is currently in English, compared to nearly 100% just ten years ago. Consequently, there is a tremendous and increasing need for language processing tools that make this information accessible to users of different languages. Among these linguistic tools we mention information search and retrieval; filtering, indexing and classification; summarization; translation; text-tospeech and dictation, etc. Recent advances in computational linguistic research are making the development of efficient tools feasible, but it is important to remember that these tools cannot be made independent of the particular language to be treated. English language tools will simply not work for Finnish. Advanced Nordic language resources are needed The development of language-specific tools typically requires research on very extensive language resources. These comprise large text and speech collections commonly called corpora. Adequately coded and quality-controlled corpora provide the empirical basis for nearly every stage in research and development of language technology products: (i) compiling linguistic requirements and specifications for new systems, (ii) extracting linguistic knowledge in the form of word lists, grammar rules, etc. (iii) repeated testing of research stage prototypes against real language data, and (iv) final evaluation of systems and applications. Not only must corpora be very large in order to be representative, they must also be carefully encoded and enriched with linguistic descriptions. Given the massive ambiguity in natural language, raw text corpora are of limited use. To take a very simple example, in the Swedish sentence (1) it is impossible, based on word order alone, to determine what is the subject and what is the object of