Automatic Induction of Dialogue Structure from the Companions Dialogue Corpus

We present a mechanism to learn repeated substructures of dialogue from an appropriately annotated dialogue corpus. We hypothesised that we could automatically segment a dialogue corpus into ‘chunks’ corresponding roughly to ‘game’-like structures. In our dialogue system we manage dialogue by employing a set of hand-crafted networks (ATNs) which we call ‘Dialogue Action Forms’ (DAFs). DAFs deterministically guide the progress of a conversation with the user, and are individuated by conversational topic—when the conversation is moved onto a new topic (either by the system or the user), a new DAF is pushed onto the stack. We hoped to automatically induce these DAFs from a dialogue corpus. To do this, we needed to derive substantive DAFs, such as might be used to learn about someone’s family (names, ages, relationships to the user, etc.), as well as meta-conversational DAFs that manage the dialogue at the ‘exchange structure’ or ‘game’ level, and we needed to combine the two. We hoped to find generalisations that clustered discovered topic chunks into classes of functionally similar chunks. To achieve this, our method was to devise a text segmentation technique specifically suitable for dialogue, and to use this in conjunction with a tool for analysing dialogue structure in terms of dialogue act sequences.