论文信息 - A Non-deterministic Tokeniser for Finite-State Parsing

A Non-deterministic Tokeniser for Finite-State Parsing

This paper describes a non-deterministic tokeniser implemented and used for the development of a French finite-state grammar. The tokeniser includes a finite-state automaton for simple tokens and a lexical transducer that encodes a wide variety of multiword expressions, associated with multiple lexical descriptions when required.

Jean-pierre Chanod

[1] Kimmo Koskenniemi,et al. Compiling and Using Finite-State Syntactic Rules , 1992, COLING.

[2] Marti A. Hearst,et al. Adaptive Sentence Boundary Disambiguation , 1994, ANLP.

[3] Steven Abney,et al. Parsing By Chunks , 1991 .

[4] Aravind K. Joshi,et al. 34th Annual Meeting of the Association for Computational Linguistics , 1996 .

[5] Lauri Karttunen,et al. Two-Level Morphology with Composition , 1992, COLING.

[6] Frederique Segond,et al. Using a Finite-State Based Formalism to Identify and Generate Multiword Expressions , 1995 .

[7] Emmanuel Roche. Analyse syntaxique transformationnelle du francais par transducteurs et lexique-grammaire , 1993 .

[8] Jean-Pierre Chanod. Finite-state Composition of French Verb Morphology Finite-state Composition of French Verb Morphology , 1994 .

[9] Max Silberztein,et al. Dictionnaires électroniques et analyse automatique de textes : le système intex , 1993 .

[10] Pasi Tapanainen,et al. What is a word, What is a sentence? Problems of Tokenization , 1994 .

[11] Lauri Karttunen. Directed Replacement , 1996, ACL.