From grammar-based MT to post-processed SL representations

Abstract This paper proposes the implementation of a post-processing stage to a grammar-based machine translation (MT) system from written into sign language for the pair written Greek–Greek sign language (GSL), where translation output is visualized by an avatar. Post-processing is applied on the output of the MT system’s transfer module by activating an editing environment for SL which utilizes a dictionary of signs coded for HamNoSys transcriptions and non-manual elements along with grammar knowledge. This post-processing environment may also be used as a standalone editing environment for expert users allowing for a considerable range of editing actions, as well as an SL editing tool for the general public who are not familiar with the underlying technology. In both cases, this environment provides the opportunity of on-the-fly structuring of new signed utterances going far beyond current limitations in composition with new signed language content.

[1]  Andy Way,et al.  An Example-Based Approach to Translating Sign Language , 2005, MTSUMMIT.

[2]  Hermann Ney,et al.  Extensions of the Sign Language Recognition and Translation Corpus RWTH-PHOENIX-Weather , 2014, LREC.

[3]  R. Elliott,et al.  An Overview of the SiGML Notation and SiGMLSigning Software System , 2004 .

[4]  Stavroula-Evita Fotinea,et al.  GSLC: Creation and Annotation of a Greek Sign Language Corpus for HCI , 2007, HCI.

[5]  W. Stokoe.  Sign Language Structure , 1980 .

[6]  Ian Marshall,et al.  Sign Language Translation via DRT and HPSG , 2002, CICLing.

[7]  Steven Abney,et al.  Parsing By Chunks , 1991 .

[8]  Bob Carpenter,et al.  The logic of typed feature structures , 1992 .

[9]  Stavroula-Evita Fotinea,et al.  Creation of a multipurpose sign language lexical resource: The GSL lexicon database , 2014 .

[10]  Στέλιος Πιπερίδης,et al.  A Unified POS Tagging Architecture and its Application to Greek , 2000 .

[11]  Martin Kay,et al.  Functional Unification Grammar: A Formalism for Machine Translation , 1984, ACL.

[12]  Eleni Efthimiou,et al.  Elicitation tasks and materials designed for Dicta-Sign's multi-lingual corpus , 2010 .

[13]  Onno Crasborn,et al.  Workshop Proceedings: 5th Workshop on the Representation and Processing of Sign Languages: Interactions between Corpus and Lexicon , 2012, LREC 2012.

[14]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[15]  Carol Neidle,et al.  SignStream™: A database tool for research on visual-gestural language , 2002 .

[16]  Rosalee Wolfe,et al.  Synthetic Corpora: A Synergy of Linguistics and Computer Animation , 2010 .

[17]  Matthew G. Snover,et al.  A Study of Translation Edit Rate with Targeted Human Annotation , 2006, AMTA.

[18]  Eleni Efthimiou,et al.  Grammar/Prosody Modelling in Greek Sign Language: Towards the Definition of Built-In Sign Synthesis Rules , 2011, Gesture Workshop.

[19]  Stavroula-Evita Fotinea,et al.  A prototype Greek text to Greek Sign Language conversion system , 2010, Behav. Inf. Technol..

[20]  N. E. Johnson,et al.  Knowledge based systems series Vol. 1: Knowledge acquisition for knowledge based systems Vol. 2: Knowledge acquisition tools for expert systems: B Gaines and J Boose (Eds) , 1990, Knowl. Based Syst..

[21]  C. Pollard,et al.  Center for the Study of Language and Information , 2022 .

[22]  Stavroula-Evita Fotinea,et al.  Processing linguistic data for GSL structure representation , 2006 .

[23]  Antonio García,et al.  Paper title: Design, Development and Field Evaluation of a Spanish into Sign Language Translation System , 2009 .

[24]  Michael Filhol,et al.  A rule triggering system for automatic text-to-sign translation , 2016, Universal Access in the Information Society.

[25]  Stuart M. Shieber,et al.  Unification and Grammatical Theory , 1986 .

[26]  Stelios Piperidis,et al.  A Robust Parser for Unrestricted Greek Text , 2000, LREC.

[27]  Eleni Efthimiou,et al.  Construction and Exploitation of Sign Language Corpora. 3rd Workshop on the Representation and Processing of Sign Languages , 2008 .

[28]  Andy Way,et al.  Combining data-driven MT systems for improved sign languagetranslation , 2007 .

[29]  Yung-Hui Lee,et al.  Taiwan sign language (TSL) recognition based on 3D data and neural networks , 2009, Expert Syst. Appl..

[30]  Georgios Kouroupetroglou,et al.  Gesture and Sign Language in Human-Computer Interaction and Embodied Communication , 2012, Lecture Notes in Computer Science.

[31]  John Glauert,et al.  Dicta-Sign – Building a Multilingual Sign Language Corpus , 2012 .

[32]  Petros Maragos,et al.  Sign Language technologies and resources of the Dicta-Sign project , 2012 .

[33]  Lynette van Zijl,et al.  The South African sign language machine translation project: issues on non-manual sign generation , 2006 .

[34]  Thomas Hanke.  HamNoSys – Representing Sign Language Data in Language Resources and Language Processing Contexts , 2004 .

[35]  Matt Huenerfauth,et al.  A Multi-Path Architecture for Machine Translation of English Text into American Sign Language Animation , 2004, NAACL.

[36]  Walter Daelemans,et al.  Proceedings of Language Resources and Evaluation Conference (LREC) 2008 , 2008 .

[37]  Dean Barker,et al.  South African Sign Language Machine Translation System , 2003, AFRIGRAPH '03.

[38]  N. Curteanu.  Book Reviews: Lecture on Contemporary Syntactic Theories: An Introduction to Unification-Based Approaches to Grammar , 1987, CL.

[39]  Norman I. Badler,et al.  A machine translation system from English to American Sign Language , 2000, AMTA.

[40]  Petros Maragos,et al.  The Dicta-Sign Wiki: Enabling Web Communication for the Deaf , 2012, ICCHP.

[41]  Ian Marshall,et al.  Grammar Development for Sign Language Avatar-Based Synthesis , 2005 .

[42]  Hermann Ney,et al.  Statistical Sign Language Translation , 2004 .

[43]  Onno Crasborn,et al.  Using ELAN for annotating sign language corpora in a team setting , 2010, LREC 2010.

[44]  Juan Carlos López,et al.  Methodology for developing an advanced communications system for the Deaf in a new domain , 2014, Knowl. Based Syst..

[45]  Stavroula-Evita Fotinea,et al.  SiS-Builder: A Sign Synthesis Support Tool. , 2010 .

[46]  Andy Way,et al.  Manual labour: tackling machine translation for sign languages , 2013, Machine Translation.

[47]  Ralph Weischedel,et al.  A STUDY OF TRANSLATION ERROR RATE WITH TARGETED HUMAN ANNOTATION , 2005 .

[48]  Mark Wells,et al.  Tessa, a system to aid communication with deaf people , 2002, ASSETS.

[49]  Andy Way,et al.  Lost in translation: the problems of using mainstream MTevaluation metrics for sign language translation , 2006 .

[50]  Paola Pietrandrea,et al.  The notation of signed texts: Open questions and indications for further research , 2002 .

[51]  Προκόπης Προκοπίδης,et al.  A suite of NLP tools for Greek , 2011 .