EmpathicSDS: Investigating Lexical and Acoustic Mimicry to Improve Perceived Empathy in Speech Dialogue Systems

In human-to-human conversations, showing empathy and thus understanding for the situation of the opposite party is crucial for a natural conversation. Thereby, emotional mimicry, i.e. imitating expressions of the person whom we are interacting with, is one of the basic mechanisms contributing to empathy. State-of-the-art speech dialogue systems still lack the ability of showing empathy, which limits naturalness. Thus, we present EmpathicSDS, a prototype to investigate the potential of lexical and acoustic mimicry for improving empathy in conversational interfaces. Our prototype comprises three different modes: 1.) neutral, where the system's response to a user query is static, 2.) lexical mimicry, where the wording of the user is reappraised by the system, and 3.) lexical and acoustic mimicry, which applies both lexical mimicry and a matching of the system's voice emotion to the user's emotional state. We conducted a user study with 33 participants to evaluate the effect of the mimicry approaches on user perception and to explore the role of user emotions. Our results show that lexical mimicry significantly improves perceived empathy and personalization without affecting efficiency. Acoustic mimicry can further improve naturalness in the condition of positive emotion while impairing efficiency in the negative condition.