Burstiness across multimodal human interaction reveals differences between verbal and non-verbal communication

Recent studies of naturalistic face-to-face communication have demonstrated temporal coordination patterns such as the synchronization of verbal and non-verbal behavior, which provides evidence for the proposal that verbal and non-verbal communicative control derives from one system. In this study, we argue that the observed relationship between verbal and non-verbal behaviors depends on the level of analysis. In a re-analysis of a corpus of naturalistic multimodal communication (Louwerse et al., 2012), we focus on measuring the temporal patterns of specific communicative behaviors in terms of their burstiness. We examined burstiness estimates across different roles of the speaker and different communicative channels. We observed more burstiness for verbal versus non-verbal channels, and for more versus less informative language sub-channels. These findings demonstrate a new method for analyzing temporal patterns in communicative behaviors, and they suggest a more complex relationship between verbal and non-verbal channels than suggested by prior studies.