Augmented transition networks (ATNs) for dialog control: A longitudinal study

ABSTRACT Our research team has implemented over a dozen spoken natural language dialog systems in varied domains over the past decade. Each system uses the same underlying dialog controller – an augmented transition network (ATN) – for maintaining a cohesive, natural conversation with the user. In this paper, we will examine the evolution of our use of ATNs, present statistical analysis of the features of our ATNs, and discuss lessons learned. KEY WORDS Dialog, natural language processing, mixed initiative, augmented transition networks, virtual humans. 1. Introduction Since approximately 1996, our research team has worked on a series of PC-based applications in which the user interacts with responsive virtual characters. Applications have ranged from trauma patient assessment [1] to learning military tank maintenance diagnostic skills [2] to gaining skills in avoiding non-response during field interviews [3]. In these applications, the computer s im ul a te pro n’ b h v .Users interact with the virtual characters via voice, mouse, menu, and/or keyboard. We are certainly not alone in developing training, assessment, marketing, and oth erv iu alm np c s( ,.g[4,5,6,7,8,9,10,11]), but the breadth across domains and the consistency of the underlying architecture allows us to m e as ur o yt’ p f nc lg id .We have developed a dialog system architecture that e nabl su r tog icp d v w hvirtual humans and see and hear their realistic responses [12]. As seen in Figure 1, among the components that underlie the architecture are a Language Processor and a Behavior Engine. The Language Processor accepts spoken input and maps this input to an underlying semantic representation, and then functions in reverse, mapping semantic representations to gestural and speech output. Our applications variously use spoken natural language interaction [2], text-based interaction, and menu-based interaction. The Behavior Engine maps Language Processor output and other environmental stimuli to virtual human behaviors. The underlying data structure of the Behavior Engine is an augmented transition network (ATN) to be described in more detail in Section 3. These behaviors include decision-making and problem solving, performing actions in the virtual world, and spoken dialog. The Behavior Engine also controls the dynamic loading of contexts and knowledge for use by the Language Processor. The virtual characters are rendered via a Visualization Engine that performs gesture, movement, and speech actions, through morphing of vertices of a 3D model and playing of key-framed animation files (largely based on motion capture data). Physical interaction with the virtual character (e.g., using medical instruments) is realized via object-based and instrument-specific selection maps [13]. These interactions are controlled by both the Behavior Engine and Visualization Engine.