Pointing and reference reconsidered

Current semantic theory on indexical expressions claims that demonstratively used indexicals such as this lack a referent-determining meaning but instead rely on an accompanying demonstration act like a pointing gesture. While this view allows to set up a sound logic of demonstratives, the direct-referential role assigned to pointing gestures has never been scrutinized thoroughly in semantics or pragmatics. We investigate the semantics and pragmatics of co-verbal pointing from a foundational perspective combining experiments, statistical investigation, computer simulation and theoretical modeling techniques in a novel manner. We evaluate various referential hypotheses with a corpus of object identification games set up in experiments in which body movement tracking techniques have been extensively used to generate precise pointing measurements. Statistical investigation and computer simulations show that especially distal areas in the pointing domain falsify the semantic direct-referential hypotheses concerning pointing gestures. As an alternative, we propose that reference involving pointing rests on a default inference which we specify using the empirical data. These results raise numerous problems for classical semantics–pragmatics interfaces: we argue for pre-semantic pragmatics in order to account for inferential reference in addition to classical post-semantic Gricean pragmatics.

[1]  D. McNeill Language and Gesture , 2000 .

[2]  G. Butterworth Pointing Is the Royal Road to Language for Babies , 2003 .

[3]  Massimo Poesio,et al.  Anaphora and Direct Reference: Empirical Evidence from Pointing , 2009 .

[4]  Bart Geurts,et al.  Good News about the Description Theory of Names , 1997, J. Semant..

[5]  Jean Carletta,et al.  Assessing Agreement on Classification Tasks: The Kappa Statistic , 1996, CL.

[6]  Laurence R. Horn,et al.  The handbook of pragmatics , 2004 .

[7]  Yihsiu Chen,et al.  Language and Gesture: Lexical gestures and lexical access: a process model , 2000 .

[8]  Adam Kendon,et al.  Language and Gesture: Language and gesture: unity or duality? , 2000 .

[9]  D. McNeill Hand and Mind: What Gestures Reveal about Thought , 1992 .

[10]  N. Enfield ‘Lip-pointing’: A discussion of form and function with reference to data from Laos , 2002 .

[11]  D. Davidson Truth and meaning , 2004, Synthese.

[12]  Thies Pfeiffer,et al.  Sprach-Gestik Experimente mit IADE, dem Interactive Augmented Data Explorer , 2006 .

[13]  J. D. Ruiter The production of gesture and speech , 2000 .

[14]  C. I. Lewis,et al.  The Semantic Conception of Truth and the Foundations of Semantics , 1944 .

[15]  Willard Van Orman Quine,et al.  Word and Object , 1960 .

[16]  David Braun,et al.  Structured characters and complex demonstratives , 1994 .

[17]  H. H. Clark Pointing and placing. , 2003 .

[18]  K. Jon Barwise,et al.  The situation in logic , 1989, CSLI lecture notes series.

[19]  Colin McGinn,et al.  The mechanism of reference , 1981, Synthese.

[20]  Thies Pfeiffer,et al.  Interaction between Speech and Gesture: Strategies for Pointing to Distant Objects , 2011, Gesture Workshop.

[21]  A new method of estimation of interobserver variation and its application to the radiological assessment of osteoarthrosis in hip joints. , 1988, Statistics in medicine.

[22]  Henk Zeevat,et al.  Demonstratives in Discourse , 1999, J. Semant..

[23]  Lawrence D. Roberts How Reference Works: Explanatory Models for Indexicals, Descriptions, and Opacity , 1993 .

[24]  Howard K. Wettstein,et al.  Themes from Kaplan , 1989 .

[25]  E. Lepore,et al.  Donald Davidson's truth-theoretic semantics , 2007 .

[26]  Philip R. Cohen,et al.  Intentions in Communication. , 1992 .

[27]  Thies Pfeiffer,et al.  Understanding multimodal deixis with gaze and gesture in conversational interfaces , 2011 .

[28]  M. Studdert-Kennedy Hand and Mind: What Gestures Reveal About Thought. , 1994 .

[29]  G. Butterworth,et al.  How the eyes, head and hand serve definite reference , 2000 .

[30]  Robyn Carston,et al.  Relevance Theory—New Directions and Developments , 2008 .

[31]  A. Kendon,et al.  Pointing by Hand in “Neapolitan” , 2003 .

[32]  D. Sperber,et al.  Relevance: Communication and Cognition , 1997 .

[33]  andy. luecking,et al.  Assessing Reliability on Annotations (2): Statistical Results for the deikon Scheme , 2006 .

[34]  Ines Gloeckner,et al.  Relevance Communication And Cognition , 2016 .

[35]  J. Haviland Language and Gesture: Pointing, gesture spaces, and mental maps , 2000 .

[36]  Ipke Wachsmuth,et al.  Deictic object reference in task-oriented dialogue , 2006 .

[37]  Daniel M. Oppenheimer,et al.  Accuracy in detecting referents of pointing gestures unaccompanied by language , 2006 .

[38]  B. Russell II.—On Denoting , 1905 .

[39]  Emiel Krahmer,et al.  Generating Multimodal References , 2007 .

[40]  A. Feinstein,et al.  High agreement but low kappa: I. The problems of two paradoxes. , 1990, Journal of clinical epidemiology.

[41]  Thies Pfeiffer,et al.  Measuring and Reconstructing Pointing in Visual Contexts , 2006 .

[42]  Keith Devlin,et al.  Logic and information , 1991 .

[43]  Alfred Tarski,et al.  Der Wahrheitsbegriff in den formalisierten Sprachen , 1935 .

[44]  H. Grice Logic and conversation , 1975 .

[45]  Robyn Carston,et al.  The semantics/pragmatics distinction: a view from Relevance Theory , 1998 .

[46]  Philip R. Cohen,et al.  Intentions in Communication , 1991, CL.

[47]  Philip R. Cohen,et al.  On the Unification of Speech Act Theory and Formal Semantics , 2003 .

[48]  David Kaplan,et al.  On the logic of demonstratives , 1979, J. Philos. Log..

[49]  J. Perry,et al.  Thought without Representation , 1986 .

[50]  Kensy Cooperrider,et al.  Nose-pointing: Notes on a facial gesture of Papua New Guinea , 2012 .

[51]  Werner Deutsch,et al.  The Development of Verbal and Nonverbal Devices for Reference. , 1982 .

[52]  H. Rieser,et al.  Statistical Support for the Study of Structures in Multi-Modal Dialogue : Inter-Rater Agreement and Synchronization , 2004 .

[53]  W. Levelt,et al.  Speaking: From Intention to Articulation , 1990 .

[54]  A. Bangerter,et al.  Using Pointing and Describing to Achieve Joint Focus of Attention in Dialogue , 2004, Psychological science.

[55]  H. H. Clark,et al.  Changing Ideas about Reference , 2004 .

[56]  Gennaro Chierchia,et al.  Meaning and grammar (2nd ed.): an introduction to semantics , 2000 .

[57]  Georgios Kouroupetroglou,et al.  Gesture and Sign Language in Human-Computer Interaction and Embodied Communication , 2012, Lecture Notes in Computer Science.

[58]  Saul A. Kripke,et al.  Naming and Necessity , 1980 .

[59]  David Braun Demonstratives and Their Linguistic Meanings , 1996 .

[60]  A. Kendon Gesticulation and Speech: Two Aspects of the Process of Utterance , 1981 .

[62]  D. Davidson Truth and Meaning , 1967 .

[63]  François Recanati,et al.  Pragmatics and Semantics , 2008 .

[64]  Gennaro Chierchia,et al.  Meaning and Grammar: An Introduction to Semantics , 1990 .

[65]  Jacob Cohen A Coefficient of Agreement for Nominal Scales , 1960 .

[66]  George Powell The referential-attributive distinction: A cognitive account , 2001 .

[67]  Ipke Wachsmuth,et al.  Deixis: How to Determine Demonstrated Objects Using a Pointing Cone , 2005, Gesture Workshop.

[68]  S. Kita Pointing: Where language, culture, and cognition meet , 2003 .

[69]  A. Kendon Gesture: Visible Action as Utterance , 2004 .