Evaluation of Natural Language and Speech Tools for Italian

The aim of the EVALITA Parsing Task (EPT) is at defining and extending Italian state-of-the-art parsing by encouraging the application of existing models and approaches, comparing paradigms and annotation formats. Therefore, in all the editions, held respectively in 2007, 2009 and 2011, the Task has been organized around two tracks, namely Dependency Parsing and Constituency Parsing, exploiting the same data sets made available by the organizers in two different formats. This paper describes the Dependency Parsing Task assuming an historical perspective, but mainly focussing on the last edition held in 2011. It presents and compares the resources exploited for development and testing, the participant systems and the results, showing also the improvement of resources and scores during the three editions of this contest.

[1]  Tommaso Caselli,et al.  Rule-Based Creation of TimeML Documents from Dependency Trees , 2011, AI*IA.

[2]  Hiroshi Maruyama,et al.  Structural Disambiguation With Constraint Propagation , 1990, ACL.

[3]  Fabio Brugnara,et al.  A baseline for the transcription of Italian broadcast news , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[4]  Mary P. Harper,et al.  Extensions to constraint dependency parsing for spoken language processing , 1995, Comput. Speech Lang..

[5]  Mazzei Alessandro,et al.  The Evalita 2011 Parsing Task: the Dependency Track , 2012 .

[6]  Hermann Ney,et al.  Joint-sequence models for grapheme-to-phoneme conversion , 2008, Speech Commun..

[7]  Jean-Luc Gauvain,et al.  On the Use of MLP Features for Broadcast News Transcription , 2008, TSD.

[8]  Michael C. McCord,et al.  Slot Grammars , 1980, CL.

[9]  Fabio Brugnara,et al.  Advances in automatic transcription of Italian broadcast news , 2000, INTERSPEECH.

[10]  Jean-Luc Gauvain,et al.  Lightly supervised and unsupervised acoustic model training , 2002, Comput. Speech Lang..

[11]  Giuseppe Attardi,et al.  Experiments with a Multilanguage Non-Projective Dependency Parser , 2006, CoNLL.

[12]  Leonardo Lesmo.  The Turin University Parser at Evalita 2009 , 2009 .

[13]  Massimiliano Ciaramita,et al.  Supersense Tagging of Unknown Nouns in WordNet , 2003, EMNLP.

[14]  Richard M. Schwartz,et al.  Unsupervised versus supervised training of acoustic models , 2008, INTERSPEECH.

[15]  Holger Schwenk,et al.  Continuous space language models , 2007, Comput. Speech Lang..

[16]  Frantisek Grézl,et al.  Optimizing bottle-neck features for lvcsr , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[17]  Cristina Bosco,et al.  Treebank Development: the TUT Approach , 2002 .

[18]  Ralph Debusmann,et al.  Extensible Dependency Grammar: A New Methodology , 2004, Workshop On Recent Advances In Dependency Grammar.

[19]  Chin-Hui Lee,et al.  Maximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chains , 1994, IEEE Trans. Speech Audio Process..

[20]  Hermann Ney,et al.  Unsupervised training of acoustic models for large vocabulary continuous speech recognition , 2005, IEEE Transactions on Speech and Audio Processing.

[21]  Joakim Nivre,et al.  An Efficient Algorithm for Projective Dependency Parsing , 2003, IWPT.

[22]  Cristina Bosco,et al.  Annotation Schema Oriented Validation for Dependency Parsing Evaluation , 2010 .

[23]  Jean-Luc Gauvain,et al.  Partitioning and transcription of broadcast news data , 1998, ICSLP.

[24]  Jean-Luc Gauvain,et al.  Towards task-independent speech recognition , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[25]  Jean-Luc Gauvain,et al.  Training Neural Network Language Models on Very Large Corpora , 2005, HLT.

[26]  H Hermansky,et al.  Perceptual linear predictive (PLP) analysis of speech. , 1990, The Journal of the Acoustical Society of America.

[27]  Fabio Brugnara,et al.  Cross-task portability of a broadcast news speech recognition system , 2002, Speech Commun..

[28]  Andreas Stolcke,et al.  Using MLP features in SRI's conversational speech recognition system , 2005, INTERSPEECH.

[29]  Cristina Bosco,et al.  Evalita'09 Parsing Task: comparing dependency parsers and treebanks , 2009 .

[30]  Jean-Luc Gauvain,et al.  The LIMSI Broadcast News transcription system , 2002, Speech Commun..

[31]  Wolfgang Menzel,et al.  Decision Procedures for Dependency Parsing Using Graded Constraints , 1998 .

[32]  Jason Eisner,et al.  Three New Probabilistic Models for Dependency Parsing: An Exploration , 1996, COLING.