Generating Quantified Referring Expressions with Perceptual Cost Pruning

We model the production of quantified referring expressions (QREs) that identify collections of visual items. To address this task, we propose a method of perceptual cost pruning, which consists of two steps: (1) determine what subset of quantity information can be perceived given a time limit t, and (2) apply a preference order based REG algorithm (e.g., the Incremental Algorithm) to this reduced set of information. We demonstrate that this method successfully improves the human-likeness of the IA in the QRE generation task and successfully models human-generated language in most cases.

[1]  G. Mandler,et al.  Subitizing: an analysis of its component processes. , 1982, Journal of experimental psychology. General.

[2]  R Rothkegel,et al.  Subitizing and its subprocesses , 2000, Psychological research.

[3]  Kees van Deemter,et al.  Generating Expressions that Refer to Visible Objects , 2013, NAACL.

[4]  Kees van Deemter,et al.  Natural Reference to Objects in a Visual Domain , 2010, INLG.

[5]  Robert Dale,et al.  Computational Interpretations of the Gricean Maxims in the Generation of Referring Expressions , 1995, Cogn. Sci..

[6]  Kees van Deemter Computational Models of Referring: A Study in Cognitive Science , 2016 .

[7]  Gordon Briggs,et al.  Elicitation of Quantified Description Under Time Constraints , 2019, CogSci.

[8]  Z. Pylyshyn,et al.  Why are small and large numbers enumerated differently? A limited-capacity preattentive stage in vision. , 1994, Psychological review.

[9]  Emiel Krahmer,et al.  Computational Generation of Referring Expressions: A Survey , 2012, CL.

[10]  Kees van Deemter,et al.  Generation of Quantified Referring Expressions: Evidence from Experimental Data , 2013, ENLG.

[11]  C. Gallistel,et al.  The Child's Understanding of Number , 1979 .

[12]  Gordon Briggs,et al.  A Computational Model of the Role of Attention in Subitizing and Enumeration , 2017, CogSci.

[13]  Micha Elsner,et al.  Visual Complexity and Its Effects on Referring Expression Generation. , 2018, Cognitive science.

[14]  E. Spelke,et al.  The construction of large number representations in adults , 2003, Cognition.

[15]  E. L. Kaufman,et al.  The discrimination of visual number. , 1949, The American journal of psychology.

[16]  M. Elsner,et al.  Where's Wally: the influence of visual salience on referring expression generation , 2013, Front. Psychol..

[17]  Justin Halberda,et al.  Grouping by proximity and the visual impression of approximate number in random dot arrays , 2016, Vision Research.

[18]  Henry Railo,et al.  The role of attention in subitizing , 2008, Cognition.