An Evaluation of Native versus Foreign Communicative Interactions on Users’ Behavioral Reactions towards Affective Virtual Crowds

This investigation compared the impact on the users’ non-verbal behaviors elicited by interacting with a crowd of emotional virtual humans (VHs) in native and non-native language settings. In a between-subject experiment, we collected objective metrics regarding the users’ behaviors during interaction with a crowd of VHs who expressed verbal and non-verbal communicative behaviors in response to users’ speech-based interaction. The study presented four VH crowd conditions based on their emotional disposition, namely Positive, Negative, Neutral, and Mixed, in accordance with which VHs exhibited the appropriate behaviors. Participants were tasked with collecting items in a virtual flea market, and they interacted using natural speech-based dialogue with the VHs. The language conditions were collected in the USA and under two different conditions in Taiwan. The participants in the USA group interacted with the VHs in English (a native language for the USA setting); and two different groups in Taiwan interacted with the VHs in either a foreign (English) or native (Mandarin) language, respectively. Results reveal that in the (TMA, TEN) and (UEN, TEN) conditions, native versus non-native language communication can alter social behaviors (e.g., interaction time) of participants towards virtual interlocutors. Our results also revealed strong effects of cultural differences on participants’ non-verbal social behaviors with VHs in the UEN and TMA conditions.