Multilingual Natural Language Generation for 3D Learning Environments

It is crucial for knowledge-based learning environments to be able to provide students with dynamic, realtime explanations. Achieving this requires the integration of 3D animated explanations with speech. Unfortunately, planning the integrated creation of 3D animation and the types of linguistic utterances capable of reactively and dynamically explaining them requires a wide range of knowledge about 3D models, pedagogical strategies, student models and linguistic utterances that can describe what is seen onscreen at any moment. Furthermore, the development costs associated with state-of-the-art technologies currently limit the introduction of these learning environments to monolingual systems. We present a multilingual framework for generating multimodal explanations of complex physical phenomena that combines 3D animated technologies with multilingual speech. This framework achieves multilingual generation without requiring any changes to be made to the underlying knowledge base that contains the domain knowledge.