Grounding Semantics in Olfactory Perception

Multi-modal semantics has relied on feature norms or raw image data for perceptual input. In this paper we examine grounding semantic representations in olfactory (smell) data, through the construction of a novel bag of chemical compounds model. We use standard evaluations for multi-modal semantics, including measuring conceptual similarity and cross-modal zero-shot learning. To our knowledge, this is the first work to evaluate semantic similarity on representations grounded in olfactory data.

[1]  Max M. Louwerse,et al.  Symbol Interdependency in Symbolic and Embodied Cognition , 2011, Top. Cogn. Sci..

[2]  Angeliki Lazaridou,et al.  Is this a wampimuk? Cross-modal mapping between distributional semantics and the visual world , 2014, ACL.

[3]  Marc'Aurelio Ranzato,et al.  DeViSE: A Deep Visual-Semantic Embedding Model , 2013, NIPS.

[4]  D. T. Stanton,et al.  Identification of latent variables in a semantic odor profile database using principal component analysis. , 2006, Chemical senses.

[5]  Elia Bruni,et al.  Multimodal Distributional Semantics , 2014, J. Artif. Intell. Res..

[6]  Darren R. Gitelman,et al.  When the Sense of Smell Meets Emotion: Anxiety-State-Dependent Olfactory Processing and Neural Circuitry Adaptation , 2013, The Journal of Neuroscience.

[7]  James M. Bower,et al.  Quantifying Olfactory Perception: Mapping Olfactory Perception Space by Using Multidimensional Scaling and Self-Organizing Maps , 2002, Neurocomputing.

[8]  J. Price,et al.  Central olfactory connections in the macaque monkey , 1994, The Journal of comparative neurology.

[9]  Felix Hill,et al.  SimLex-999: Evaluating Semantic Models With (Genuine) Similarity Estimation , 2014, CL.

[10]  Patrick Pantel,et al.  From Frequency to Meaning: Vector Space Models of Semantics , 2010, J. Artif. Intell. Res..

[11]  Quoc V. Le,et al.  Grounded Compositional Semantics for Finding and Describing Images with Sentences , 2014, TACL.

[12]  Thomas Martinetz,et al.  On the dimensions of the olfactory perception space , 2004, Neurocomputing.

[13]  R. Axel,et al.  A novel multigene family may encode odorant receptors: A molecular basis for odor recognition , 1991, Cell.

[14]  Sabine Schulte im Walde,et al.  A Multimodal LDA Model integrating Textual, Cognitive and Visual Modalities , 2013, EMNLP.

[15]  Yansong Feng,et al.  Visual Information in Semantic Representation , 2010, NAACL.

[16]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[17]  Rada Mihalcea,et al.  Going Beyond Text: A Hybrid Image-Text Approach for Measuring Word Relatedness , 2011, IJCNLP.

[18]  Felix Hill,et al.  Learning Abstract Concept Embeddings from Multi-Modal Data: Since You Probably Can’t See What I Mean , 2014, EMNLP.

[19]  Stephen Clark,et al.  Improving Multi-Modal Representations Using Image Dispersion: Why Less is Sometimes More , 2014, ACL.

[20]  Stephen Clark,et al.  From distributional semantics to feature norms: grounding semantic models in human perceptual data , 2015, IWCS.

[21]  Carina Silberer,et al.  Grounded Models of Semantic Representation , 2012, EMNLP.

[22]  Stephen Clark,et al.  Vector Space Models of Lexical Meaning , 2015 .

[23]  Léon Bottou,et al.  Learning Image Embeddings using Convolutional Neural Networks for Improved Multi-Modal Semantics , 2014, EMNLP.