Audience size and contextual effects on information density in Twitter conversations

The “uniform information density” (UID) hypothesis proposes that language producers aim for a constant rate of information flow within a message, and research on monologue-like written texts has found evidence for UID in production. We consider conversational messages, using a large corpus of tweets, and look for UID behavior. We do not find evidence of UID behavior, and even find context effects that are opposite that of previous, monologue-based research. We propose that a more collaborative conception of information density and careful consideration of channel noise may be needed in the informationtheoretic framework for conversation.

[1]  Ting Qian,et al.  Cue Effectiveness in Communicatively Efficient Discourse Production , 2012, Cogn. Sci..

[2]  Alice Turk,et al.  The Smooth Signal Redundancy Hypothesis: A Functional Explanation for Relationships between Redundancy, Prosodic Prominence, and Duration in Spontaneous Speech , 2004, Language and speech.

[3]  Michael C. Frank,et al.  Shared common ground influences information density in microblog texts , 2015, NAACL.

[4]  Eugene Charniak,et al.  Entropy Rate Constancy in Text , 2002, ACL.

[5]  R. Levy.  Expectation-based syntactic comprehension , 2008, Cognition.

[6]  E. Schegloff,et al.  Opening up Closings , 1973 .

[7]  Telephone Goodbyes.  Telephone Goodbyes , 2008 .

[8]  Vera Demberg,et al.  Syntactic Surprisal Affects Spoken Word Duration in Conversational Contexts , 2012, EMNLP.

[9]  Shoichi Iwasaki,et al.  The Northridge earthquake conversations: The floor structure and the ‘loop’ sequence in Japanese conversation☆ , 1997 .

[10]  Eugene Charniak,et al.  Variation of Entropy and Parse Trees of Sentences as a Function of the Sentence Number , 2003, EMNLP.

[11]  Brendan T. O'Connor,et al.  Improved Part-of-Speech Tagging for Online Conversational Text with Word Clusters , 2013, NAACL.

[12]  T. Florian Jaeger,et al.  Redundancy and reduction: Speakers manage syntactic information density , 2010, Cognitive Psychology.

[13]  Gabriel Doyle,et al.  Mapping Dialectal Variation by Querying Social Media , 2014, EACL.

[14]  M. Aylett,et al.  Language redundancy predicts syllabic duration and the spectral characteristics of vocalic syllable nuclei. , 2006, The Journal of the Acoustical Society of America.

[15]  Christopher Potts,et al.  Goal-Driven Answers in the CardsDialogue Corpus , 2012 .

[16]  T. Jaeger,et al.  Speaking Rationally: Uniform Information Density as an Optimal Strategy for Language Production , 2008 .

[17]  Dan Jurafsky,et al.  Effects of disfluencies, predictability, and utterance position on word form variation in English conversation. , 2003, The Journal of the Acoustical Society of America.

[18]  Nigel Ward,et al.  Looking for Entropy Rate Constancy in Spoken Dialog , 2009 .

[19]  N. Arnett.  Goal-driven Answers in the Cards Dialogue Corpus , 2012 .

[20]  Thomas Hofmann,et al.  Speakers optimize information density through syntactic reduction , 2007 .

[21]  V. Yngve.  On getting a word in edgewise , 1970 .

[22]  E. Schegloff.  Discourse as an interactional achievement : Some uses of "Uh huh" and other things that come between sentences , 1982 .

[23]  H. H. Clark,et al.  Collaborating on contributions to conversations , 1987 .