Modular definition of multimodal ECA communication acts to improve dialogue robustness and depth of intention

In this paper we propose a modular structure to define communication acts with verbal and nonverbal elements inspired in the SAIBA model. Our modular structure is a conceptual interpretation of the functional features of a multimodal interaction platform we have developed, with an embodied conversational agent (ECA) that implements verbal and gestural communication strategies aimed at minimising robustness and fluency problems that are typically encountered in spoken language dialogue systems (SLDSs). We conclude that it is useful to add a pre-verbal level on top of the FML-BML scheme in the SAIBA framework, and we propose a category extension for FML to account for communication elements that have to do with the speaker’s non-declared intentions.