Audiovisual Correlates of Interrogativity: A Comparative Analysis of Catalan and Dutch

Languages employ different strategies to mark an utterance as a polar (yes–no) question, including syntax, intonation and gestures. This study analyzes the production and perception of information-seeking questions and broad focus statements in Dutch and Catalan. These languages use intonation for marking questionhood, but Dutch also exploits syntactic variation for this purpose. A production task revealed the expected language-specific auditory differences, but also showed that gaze and eyebrow-raising are used in this distinction. A follow-up perception experiment revealed that perceivers relied greatly on auditory information in determining whether an utterance is a question or a statement, but accuracy was further enhanced when visual information was added. Finally, the study demonstrates that the concentration of several response-mobilizing cues in a sentence is positively correlated with the perceivers’ ratings of these utterances as interrogatives.

[1]  M. Swerts,et al.  Perceiving incredulity: The role of intonation and facial gestures , 2013 .

[2]  Heather L. Balog,et al.  Do children produce the melody before the words? A review of developmental intonation research , 2002 .

[3]  S. Levinson.  Questions and responses in YéliDnye, the Papuan language of Rossel Island , 2010 .

[4]  Judith Haan.  Speaking of Questions , 2002 .

[5]  Teresa Cabré,et al.  Intonational phonology of Catalan and its dialectal varieties , 2015 .

[6]  T. Stivers,et al.  An overview of the question-response system in American English conversation , 2010 .

[7]  R. Rooij,et al.  On Polar Questions , 2003 .

[8]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[9]  Donald W. Fiske,et al.  Face-to-face interaction: Research, methods, and theory , 1977 .

[10]  Daniel C. Richardson,et al.  Conversation, Gaze Coordination, and Beliefs About Visual Context , 2009, Cogn. Sci..

[11]  M. Argyle,et al.  Gaze and Mutual Gaze , 1994, British Journal of Psychiatry.

[12]  Elina Weiste,et al.  Therapeutic Discourse , 2015 .

[13]  Tanya Stivers,et al.  Mobilizing Response , 2010 .

[14]  H. Vilhjálmsson.  Autonomous communicative behaviors in avatars , 1997 .

[15]  Pilar Prieto,et al.  ‘Seeing tunes.’ The role of visual gestures in tune interpretation , 2011 .

[16]  W. Horton,et al.  Conversational Grounding in Younger and Older Adults: The Effect of Partner Visibility and Referent Abstractness in Task-Oriented Dialogue , 2012 .

[17]  María L. Flecha-García,et al.  Eyebrow raises in dialogue and their relation to discourse structure, utterance function and pitch accents in English , 2010, Speech Commun..

[18]  Emiel Krahmer,et al.  Guess Who? An Interactive and Entertaining Game-Like Platform for Investigating Human Emotions , 2011, HCI.

[19]  J.J.M. Haan-van Ditzhuyzen.  Speaking of questions: an exploration of Dutch question intonation , 2001 .

[20]  C. Englert.  Questions and responses in Dutch conversations , 2010 .

[21]  D. Massaro,et al.  Perceiving Prosody from the Face and Voice: Distinguishing Statements from Echoic Questions in English , 2003, Language and speech.

[22]  M. Hayashi,et al.  Transformative answers: One way to resist a question's constraints , 2010 .

[23]  Anita M. Pomerantz,et al.  Telling My Side: “Limited Access’ as a “Fishing” Device , 1980 .

[24]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[25]  Alan Cruttenden,et al.  Falls and rises: meanings and universals , 1981, Journal of Linguistics.

[26]  A. Kendon.  Some functions of gaze-direction in social interaction. , 1967, Acta psychologica.

[27]  P. Prieto,et al.  The Syntax-Prosody Interface: Catalan interrogative sentences headed by que , 2007 .

[28]  Elisabetta Bevacqua,et al.  A Model of Attention and Interest Using Gaze Behavior , 2005, IVA.

[29]  Annie Rialland.  Question prosody : an African perspective , 2007 .

[30]  Federico Rossano,et al.  Questioning and responding in Italian , 2010 .

[31]  A. Kendon.  Conducting Interaction: Patterns of Behavior in Focused Encounters , 1990 .

[32]  Dwight L. Bolinger,et al.  Intonation and Its Uses: Melody in Grammar and Discourse , 1989 .

[33]  Ronald Geluykens,et al.  On the myth of rising intonation in polar questions , 1988 .

[34]  Emanuel A. Schegloff,et al.  Commentary on Stivers and Rossano: “Mobilizing Response” , 2010 .

[35]  David House,et al.  Perception of question intonation and facial gestures , 2007 .