We went to look for meaning and all we got were these lousy representations: aspects of meaning representation for computational semantics

In this paper we examine different meaning representations that are commonly used in different natural language applications today and discuss their limits, both in terms of the aspects of the natural language meaning they are modelling and in terms of the aspects of the application for which they are used.

[1]  ScienceOpen Admin.  Glossa: a journal of general linguistics , 2018 .

[2]  Yulia Tsvetkov,et al.  A bottom up approach to category mapping and meaning change , 2015, NetWordS.

[3]  José M. F. Moura,et al.  Visual Dialog , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Jörg Tiedemann,et al.  Continuous multilinguality with language vectors , 2016, EACL.

[5]  Pentti Haddington.  The organization of gaze and assessments as resources for stance taking , 2006 .

[6]  Per Linell,et al.  Meaning Potentials and the Interaction between Lexis and Contexts: An Empirical Substantiation , 2007 .

[7]  M. Gullberg,et al.  What’s New? Gestures Accompany Inferable Rather Than Brand-New Referents in Discourse , 2020, Frontiers in Psychology.

[8]  ChengXiang Zhai,et al.  Instance Weighting for Domain Adaptation in NLP , 2007, ACL.

[9]  Jan van Eijck,et al.  Computational Semantics with Functional Programming , 2010 .

[10]  Elia Bruni,et al.  The PhotoBook Dataset: Building Common Ground through Visually-Grounded Dialogue , 2019, ACL.

[11]  Francesc Moreno-Noguer,et al.  Combining Geometric, Textual and Visual Features for Predicting Prepositions in Image Descriptions , 2015, EMNLP.

[12]  Robin Cooper,et al.  Records and Record Types in Semantic Theory , 2005, J. Log. Comput..

[13]  Karl Stratos,et al.  Midge: Generating Image Descriptions From Computer Vision Detections , 2012, EACL.

[14]  Johan Bos,et al.  Predicate logic unplugged , 1996 .

[15]  Dan Flickinger,et al.  Minimal Recursion Semantics: An Introduction , 2005 .

[16]  Lei Zhang,et al.  Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[17]  Henrik I. Christensen,et al.  Situated Dialogue and Spatial Organization: What, Where… and Why? , 2007 .

[18]  Daniel Bonevac.  Discourse Representation Theory , 2012 .

[19]  L. Stark,et al.  Spontaneous Eye Movements During Visual Imagery Reflect the Content of the Visual Scene , 1997, Journal of Cognitive Neuroscience.

[20]  Guillaume Lample,et al.  Word Translation Without Parallel Data , 2017, ICLR.

[21]  Terry Regier,et al.  The Human Semantic Potential: Spatial Language and Constrained Connectionism , 1996 .

[22]  Nazli Ikizler-Cinbis,et al.  Automatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures , 2016, J. Artif. Intell. Res..

[23]  Jacob Andreas,et al.  Experience Grounds Language , 2020, EMNLP.

[24]  Guillaume Lample,et al.  What you can cram into a single $&!#* vector: Probing sentence embeddings for linguistic properties , 2018, ACL.

[25]  Danqi Chen,et al.  of the Association for Computational Linguistics: , 2001 .

[26]  Katrin Erk,et al.  How to marry a star: Probabilistic constraints for meaning in context , 2020, SCIL.

[27]  Richard Montague,et al.  The Proper Treatment of Quantification in Ordinary English , 1973 .

[28]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[29]  Uri Hadar,et al.  The Semantic Specificity of Gesture , 2004 .

[30]  David P. Wilkins.  Adam Kendon (2004). Gesture: Visible action as utterance , 2006 .

[31]  P. Petta,et al.  Computational models of emotion , 2010 .

[32]  Stephen Clark,et al.  Mathematical Foundations for a Compositional Distributional Model of Meaning , 2010, ArXiv.

[33]  Xuri Tang,et al.  A state-of-the-art of semantic change computation , 2018, Natural Language Engineering.

[34]  Alex Lascarides,et al.  Logics of Conversation , 2005, Studies in natural language processing.

[35]  Joshua B. Tenenbaum,et al.  Building machines that learn and think like people , 2016, Behavioral and Brain Sciences.

[36]  Lea Fleischer,et al.  The Senses Considered As Perceptual Systems , 2016 .

[37]  C. Howes,et al.  Integrating laughter into spoken dialogue systems: preliminary analysis and suggested programme , 2018, FAIM/ISCA Workshop on Artificial Intelligence for Multimodal Human Robot Interaction.

[38]  Timothy Baldwin,et al.  An Empirical Evaluation of doc2vec with Practical Insights into Document Embedding Generation , 2016, Rep4NLP@ACL.

[39]  Sotaro Kita,et al.  What does cross-linguistic variation in semantic coordination of speech and gesture reveal? Evidence for an interface representation of spatial thinking and speaking , 2003 .

[40]  John D. Kelleher,et al.  Exploring the Functional and Geometric Bias of Spatial Relations Using Neural Language Models , 2018 .

[41]  Tomek Strzalkowski,et al.  From Discourse to Logic , 1991 .

[42]  Douwe Kiela,et al.  SentEval: An Evaluation Toolkit for Universal Sentence Representations , 2018, LREC.

[43]  Jure Leskovec,et al.  Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change , 2016, ACL.

[44]  Vidya Somashekarappa,et al.  An Annotation Approach for Social and Referential Gaze in Dialogue , 2020, LREC.

[45]  David Schlangen,et al.  MeetUp! A Corpus of Joint Activity Dialogues in a Visual Environment , 2019, ArXiv.

[46]  Nora Hollenstein,et al.  Sequence labelling and sequence classification with gaze: Novel uses of eye-tracking data for Natural Language Processing , 2020, Lang. Linguistics Compass.

[47]  José M. F. Moura,et al.  Visual Coreference Resolution in Visual Dialog using Neural Module Networks , 2018, ECCV.

[48]  Simao Herdade,et al.  Image Captioning: Transforming Objects into Words , 2019, NeurIPS.

[49]  B. Laeng,et al.  Tracking down the path of memory: eye scanpaths facilitate retrieval of visuospatial information , 2015, Cognitive Processing.

[50]  Staffan Larsson,et al.  Issue-based Dialogue Management , 2002 .

[51]  Hugh F. Durrant-Whyte,et al.  A solution to the simultaneous localization and map building (SLAM) problem , 2001, IEEE Trans. Robotics Autom..

[52]  Staffan Larsson,et al.  Probabilistic Type Theory and Natural Language Semantics , 2015, LILT.

[53]  Simon Dobnik,et al.  When an Image Tells a Story: The Role of Visual and Semantic Information for Generating Paragraph Descriptions , 2020, INLG.

[54]  Francis Jeffry Pelletier,et al.  Representation and Inference for Natural Language: A First Course in Computational Semantics , 2005, Computational Linguistics.

[55]  Thierry Poibeau,et al.  Multi-SimLex: A Large-Scale Evaluation of Multilingual and Crosslingual Lexical Semantic Similarity , 2020, Computational Linguistics.

[56]  Qi Wu,et al.  Vision-and-Language Navigation: Interpreting Visually-Grounded Navigation Instructions in Real Environments , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[57]  A. Blank.  Why do new meanings occur? A cognitive typology of the motivations for lexical semantic change , 1999 .

[58]  H. Paul,et al.  Principles of the history of language , 1888 .

[59]  Eneko Agirre,et al.  A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings , 2018, ACL.

[60]  Roel M. Willems,et al.  When language meets action: the neural integration of gesture and speech. , 2007, Cerebral cortex.

[61]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[62]  Stefan Lee,et al.  Embodied Question Answering , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[63]  Andy Lücking,et al.  Modeling co-verbal gesture perception in Type Theory with Records , 2016, 2016 Federated Conference on Computer Science and Information Systems (FedCSIS).

[64]  Laura M. Morett,et al.  Semantic Relationships Between Representational Gestures and Their Lexical Affiliates Are Evaluated Similarly for Speech and Text , 2020, Frontiers in Psychology.

[65]  Marco Baroni,et al.  Human few-shot learning of compositional instructions , 2019, CogSci.

[66]  Jiasen Lu,et al.  Hierarchical Question-Image Co-Attention for Visual Question Answering , 2016, NIPS.

[67]  Mehdi Ghanimifard,et al.  Learning to Compose Spatial Relations with Grounded Neural Language Models , 2017, IWCS.

[68]  Samy Bengio,et al.  Show and tell: A neural image caption generator , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[69]  Lars Borin,et al.  Survey of Computational Approaches to Diachronic Conceptual Change , 2018, ArXiv.

[70]  Richard Socher,et al.  Knowing When to Look: Adaptive Attention via a Visual Sentinel for Image Captioning , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[71]  Emily M. Bender,et al.  On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜 , 2021, FAccT.

[72]  Uwe Reyle,et al.  Dealing with Ambiguities by Underspecification: Construction, Representation and Deduction , 1993, J. Semant..

[73]  David Schlangen,et al.  Grounded Agreement Games: Emphasizing Conversational Grounding in Visual Dialogue Settings , 2019, ArXiv.

[74]  Jenny Myrendal,et al.  Word Meaning Negotiation in Online Discussion Forum Communication , 2015 .

[75]  M. Sadrzadeh,et al.  Affordance competition in dialogue: the case of syntactic universals , 2020 .

[76]  Gemma Boleda,et al.  Short-Term Meaning Shift: A Distributional Exploration , 2018, NAACL.

[77]  Raymond J. Mooney,et al.  Learning to Connect Language and Perception , 2008, AAAI.

[78]  Gregory Mills,et al.  Semantic negotiation in dialogue: the mechanisms of alignment , 2008, SIGDIAL Workshop.

[79]  Furu Wei,et al.  VL-BERT: Pre-training of Generic Visual-Linguistic Representations , 2019, ICLR.

[80]  Etienne B. Roesch,et al.  A Blueprint for Affective Computing: A Sourcebook and Manual , 2010 .

[81]  Erik Velldal,et al.  Diachronic word embeddings and semantic shifts: a survey , 2018, COLING.

[82]  R. A. Nelson,et al.  Common ground. , 2020, Lancet.

[83]  David Scott Warren,et al.  A parsing method for montague grammars , 1978 .

[84]  M. Engelmann.  The Philosophical Investigations , 2013 .

[85]  Eva Schlinger,et al.  How Multilingual is Multilingual BERT? , 2019, ACL.

[86]  Using space to represent categories: insights from gaze position , 2017, Psychological research.

[87]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[88]  Staffan Larsson,et al.  Formal semantics for perceptual classification PREPRINT VERSION , 2022 .

[89]  P. Johnson-Laird,et al.  Cognitive approaches to emotions , 2014, Trends in Cognitive Sciences.

[90]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[91]  E. Viding,et al.  Load theory of selective attention and cognitive control. , 2004, Journal of experimental psychology. General.

[92]  Mark Steedman,et al.  Wide-Coverage Semantic Representations from a CCG Parser , 2004, COLING.

[93]  Katrin Erk,et al.  Deep Neural Models of Semantic Shift , 2018, NAACL-HLT.

[94]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[95]  Staffan Larsson,et al.  Dialogue Acts and Updates for Semantic Coordination , 2017 .

[96]  Vicente Ordonez,et al.  ReferItGame: Referring to Objects in Photographs of Natural Scenes , 2014, EMNLP.

[97]  K. Scherer.  The dynamic architecture of emotion: Evidence for the component process model , 2009 .

[98]  C. Harris.  The Mystery of Ticklish Laughter , 1999, American Scientist.

[99]  Stefan Lee,et al.  ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks , 2019, NeurIPS.

[100]  H. Alshawi,et al.  The Core Language Engine , 1994 .

[101]  Jeroen Groenendijk,et al.  Dynamic predicate logic , 1991 .

[102]  Simon Dobnik.  Teaching mobile robots to use spatial words , 2009 .

[103]  Matthew R. Walter,et al.  Understanding Natural Language Commands for Robotic Navigation and Mobile Manipulation , 2011, AAAI.

[104]  Ali Farhadi,et al.  VisKE: Visual knowledge extraction and question answering by visual verification of relation phrases , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[105]  付伶俐.  打磨Using Language,倡导新理念 , 2014 .

[106]  Hugo Larochelle,et al.  GuessWhat?! Visual Object Discovery through Multi-modal Dialogue , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[107]  Margaret Mitchell,et al.  VQA: Visual Question Answering , 2015, International Journal of Computer Vision.

[108]  J. Ginzburg,et al.  Laughter as language , 2020 .

[109]  J. Firth,et al.  Papers in linguistics, 1934-1951 , 1957 .

[110]  Pushpak Bhattacharyya,et al.  Scanpath Complexity: Modeling Reading/Annotation Effort Using Gaze Information , 2018 .

[111]  Afra Alishahi,et al.  Correlating Neural and Symbolic Representations of Language , 2019, ACL.

[112]  Ronnie Cann,et al.  Language as Mechanisms for Interaction , 2016 .

[113]  Johan Bos,et al.  Compositional Semantics in Verbmobil , 1996, COLING.

[114]  Matthieu Cord,et al.  RUBi: Reducing Unimodal Biases in Visual Question Answering , 2019, NeurIPS.

[115]  Jonathan Ginzburg,et al.  The interactive stance : meaning for conversation , 2012 .