Data-driven Natural Language Generation: Making Machines Talk Like Humans Using Natural Corpora

With the significant improvements that have been seen in speech applications, the long-held goal of building machines that can have humanlike conversations has begun to seem more reachable; there exist spoken dialog systems which can now be used effectively by much of the general public. Despite these improvements, however, applications are still frequently limited by their unnatural spoken language generation. This thesis discusses the problem of human-like spoken language generation: how to make machine-generated speech more like natural human speech. The scope of this problem is large, with issues in speech synthesis, natural language generation, and spoken dialog, among other areas. The work in this thesis is primarily focused on natural language generation, with some discussion of the issues related to speech synthesis and the intersection between synthesis and language generation. In particular, we discuss a method that uses signal modifications of the synthesized waveforms to emulate what humans do when trying to be understood better while speaking in noisy conditions. One of the main differences between humanand machineproduced speech is in understandability; natural human speech is typically easier to understand. We describe a general framework, which we call uGloss, designed to improve the understandability of spoken generation of complex information. The uGloss framework employs a set of tactical generation strategies that attempt to take the expected capability of the human listener into account; by staying within those abilities the resulting spoken output is typically more easily understood. Though uGloss can improve understandability, it is not a complete solution to machine-generated human-like speech. In many other fields, from speech recognition and synthesis, to parsing and understanding, using corpus-based statistical knowledge has led to improved systems. We propose a similar data-driven approach intended to improve language generation systems, specifically for speech and dialog applications. Our proposed approach – the MOUNTAIN language generation system – is a fully-automatic approach which uses machine translation techniques to generate novel examples from a natural corpus. This system is designed to be a domain-independent method for training a generator that can produce human-like language in speech applications, by translating the machine’s internal representation of