Interactive Speech Translation in the Diplomat Project 2 Multi-engine Machine Translation

The DIPLOMAT rapid-deployment speech translation system is intended to allow na ve users to communicate across a language barrier, without strong domain restrictions, despite the error-prone nature of current speech and translation technologies. Achieving this ambitious goal depends in large part on allowing the users to interactively correct recognition and translation errors. We brieey present the Multi-Engine Machine Translation (MEMT) architecture, describing how it is well-suited for such an application. We then describe our incorporation of interactive error correction throughout the system design. We have already developed a working bidirectional Serbo-Croatian $ English system, and are currently developing Ha tian-Creole $ English and Korean $ English versions. 1 Introduction The DIPLOMAT project is designed to explore the feasibility of creating rapid-deployment, wear-able bi-directional speech translation systems. By \rapid-deployment", we mean being able to develop an MT system that performs initial translations at a useful level of quality between a new language and English within a matter of days or weeks, with continual, graceful improvement to a good level of quality over a period of months. The speech understanding component used is the Sphinx II HMM-based speaker-independent continuous speech recognition system (Huang et al., 1992; Ravishankar, 1996), with techniques for rapidly developing acoustic and language models for new languages (Rudnicky, 1995). The machine translation (MT) technology is the Multi-Engine Machine Translation (MEMT) architecture (Frederking and Nirenburg, 1994), described further below. The speech synthesis component is a newly-developed concatenative system (Lenzo, 1997) based on variable-sized compositional units. This use of subword concatenation is especially important, since it is the only currently available method for rapidly bringing up synthesis for a new language. DIPLOMAT thus involves research in MT, speech understanding and synthesis , interface design, as well as wearable computer systems. While beginning our investigations into new semi-automatic techniques for both speech and MT knowledge-base development, we have already produced an initial bidirectional system for Serbo-Croatian $ English speech translation in less than a month, and are currently developing systems. A major concern in the design of the DIPLOMAT system has been to cope with the error-prone nature of both current speech understanding and MT technology, to produce an application that is usable by non-translators with a small amount of training. We attempt to achieve this primarily through user interaction: wherever feasible, the user is presented with intermediate results, and allowed to correct them. In this paper , we will brieey describe …