Distributional Learning of Appearance

Opportunities for associationist learning of word meaning, where a word is heard or read contemperaneously with information being available on its meaning, are considered too infrequent to account for the rate of language acquisition in children. It has been suggested that additional learning could occur in a distributional mode, where information is gleaned from the distributional statistics (word co-occurrence etc.) of natural language. Such statistics are relevant to meaning because of the Distributional Principle that ‘words of similar meaning tend to occur in similar contexts’. Computational systems, such as Latent Semantic Analysis, have substantiated the viability of distributional learning of word meaning, by showing that semantic similarities between words can be accurately estimated from analysis of the distributional statistics of a natural language corpus. We consider whether appearance similarities can also be learnt in a distributional mode. As grounds for such a mode we advance the Appearance Hypothesis that ‘words with referents of similar appearance tend to occur in similar contexts’. We assess the viability of such learning by looking at the performance of a computer system that interpolates, on the basis of distributional and appearance similarity, from words that it has been explicitly taught the appearance of, in order to identify and name objects that it has not been taught about. Our experiment tests with a set of 660 simple concrete noun words. Appearance information on words is modelled using sets of images of examples of the word. Distributional similarity is computed from a standard natural language corpus. Our computation results support the viability of distributional learning of appearance.

[1]  Michael N. Jones,et al.  Perceptual Inference Through Global Lexical Similarity , 2012, Top. Cogn. Sci..

[2]  Thomas Deselaers,et al.  Visual and semantic similarity in ImageNet , 2011, CVPR 2011.

[3]  Max M. Louwerse,et al.  Symbol Interdependency in Symbolic and Embodied Cognition , 2011, Top. Cogn. Sci..

[4]  Max M. Louwerse,et al.  A Taste of Words: Linguistic Context and Perceptual Simulation Predict the Modality of Words , 2011, Cogn. Sci..

[5]  Lewis D. Griffin,et al.  Similar things look similar , 2011 .

[6]  Alexa R. Romberg,et al.  Statistical learning and language acquisition. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[7]  Jianping Fan,et al.  Leveraging loosely-tagged images and inter-object correlations for tag recommendation , 2010, ACM Multimedia.

[8]  Fei-Fei Li,et al.  What Does Classifying More Than 10, 000 Image Categories Tell Us? , 2010, ECCV.

[9]  Antonio Torralba,et al.  Semantic Label Sharing for Learning with Many Categories , 2010, ECCV.

[10]  Andrew W. Fitzgibbon,et al.  Efficient Object Category Recognition Using Classemes , 2010, ECCV.

[11]  Bernt Schiele,et al.  What helps where – and why? Semantic relatedness for knowledge transfer , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  Jianping Fan,et al.  Harvesting large-scale weakly-tagged image databases from the web , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  Dhiraj Joshi,et al.  Object Categorization: Computer and Human Vision Perspectives , 2008 .

[14]  Lewis D. Griffin,et al.  Using Basic Image Features for Texture Classification , 2010, International Journal of Computer Vision.

[15]  Sven J. Dickinson,et al.  Object Categorization: Computer and Human Vision Perspectives , 2009 .

[16]  Terrence J Sejnowski,et al.  Foundations for a New Science of Learning , 2009, Science.

[17]  Gabriella Vigliocco,et al.  Integrating experiential and distributional data to learn semantic representations. , 2009, Psychological review.

[18]  Xiaojin Zhu,et al.  Introduction to Semi-Supervised Learning , 2009, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[19]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Christoph H. Lampert,et al.  Learning to detect unseen object classes by between-class attribute transfer , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Ali Farhadi,et al.  Describing objects by their attributes , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Lewis D. Griffin,et al.  Basic Image Features (BIFs) Arising from Approximate Symmetry Type , 2009, SSVM.

[23]  Sven J. Dickinson,et al.  Object Categorization: The Evolution of Object Categorization and the Challenge of Image Abstraction , 2009 .

[24]  A. Glenberg,et al.  Symbols and Embodiment: Debates on Meaning and Cognition , 2008 .

[25]  Rich Caruana,et al.  Classification with partial labels , 2008, KDD.

[26]  M. Louwerse.  Embodied relations are encoded in language , 2008, Psychonomic bulletin & review.

[27]  Pierre Tirilly,et al.  Language modeling for bag-of-visual words image categorization , 2008, CIVR '08.

[28]  Pietro Perona,et al.  Learning and using taxonomies for fast visual categorization , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Patrick Jeuniaux,et al.  Language comprehension is both embodied and symbolic , 2008 .

[30]  T. Rogers,et al.  Where do you know what you know? The representation of semantic knowledge in the human brain , 2007, Nature Reviews Neuroscience.

[31]  Antonio Torralba,et al.  Sharing Visual Features for Multiclass and Multiview Object Detection , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Danielle S. McNamara,et al.  Handbook of latent semantic analysis , 2007 .

[33]  Paul M. B. Vitányi,et al.  The Google Similarity Distance , 2004, IEEE Transactions on Knowledge and Data Engineering.

[34]  S. Harnad.  Symbol grounding problem , 1990, Scholarpedia.

[35]  M. Kenward,et al.  An Introduction to the Bootstrap , 2007 .

[36]  Wayne D. Gray,et al.  A Proxy For All Your Semantic Needs , 2007 .

[37]  Lewis D. Griffin,et al.  Hypotheses for Image Features, Icons and Textons , 2006, International Journal of Computer Vision.

[38]  Alexander Zien,et al.  Semi-Supervised Learning , 2006 .

[39]  Luc Van Gool,et al.  Efficient, Simultaneous Detection of Multiple Object Classes , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[40]  Cordelia Schmid,et al.  Local Features and Kernels for Classification of Texture and Object Categories: A Comprehensive Study , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[41]  Frédéric Jurie,et al.  Sampling Strategies for Bag-of-Features Image Classification , 2006, ECCV.

[42]  Ted Pedersen,et al.  Using WordNet-based Context Vectors to Estimate the Semantic Relatedness of Concepts , 2006 .

[43]  Lewis D. Griffin,et al.  Optimality of the basic colour categories for classification , 2006, Journal of The Royal Society Interface.

[44]  Arthur B. Markman,et al.  Processes of Similarity Judgment , 2005, Cogn. Sci..

[45]  Antonio Criminisi,et al.  Object categorization by learned universal visual dictionary , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[46]  Alexei A. Efros,et al.  Discovering objects and their location in images , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[47]  Pietro Perona,et al.  Learning object categories from Google's image search , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[48]  Douglas L. T. Rohde,et al.  An Improved Model of Semantic Similarity Based on Lexical Co-Occurrence , 2005 .

[49]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[50]  Ted Pedersen,et al.  WordNet::Similarity - Measuring the Relatedness of Concepts , 2004, NAACL.

[51]  Jitendra Malik,et al.  Representing and Recognizing the Visual Appearance of Materials using Three-dimensional Textons , 2001, International Journal of Computer Vision.

[52]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[53]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[54]  Tat-Seng Chua,et al.  A bootstrapping approach to annotating large image collection , 2003, MIR '03.

[55]  Andrew Zisserman,et al.  Texture classification: are filter banks necessary? , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[56]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[57]  P. Bloom.  Mindreading, Communication and the Learning of Names for Things , 2002 .

[58]  K. Wilkinson,et al.  Emergent Word-Object Mapping By Children: Further Studies Using the Blank Comparison Technique , 2001 .

[59]  Michael Ramscar,et al.  Testing the Distributioanl Hypothesis: The influence of Context on Judgements of Semantic Similarity , 2001 .

[60]  Nello Cristianini,et al.  Classification using String Kernels , 2000 .

[61]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[62]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[63]  Peter Gärdenfors,et al.  Conceptual spaces - the geometry of thought , 2000 .

[64]  Peter Wiemer-Hastings,et al.  Adding syntactic information to LSA , 2000 .

[65]  Jitendra Malik,et al.  Recognizing surfaces using three-dimensional textons , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[66]  Nick Chater,et al.  Distributional Information: A Powerful Cue for Acquiring Syntactic Categories , 1998, Cogn. Sci..

[67]  Hinrich Schütze,et al.  Automatic Word Sense Discrimination , 1998, Comput. Linguistics.

[68]  Neil A. Thacker,et al.  The Bhattacharyya metric as an absolute similarity measure for frequency coded data , 1998, Kybernetika.

[69]  Vladimir Cherkassky,et al.  The Nature Of Statistical Learning Theory , 1997, IEEE Trans. Neural Networks.

[70]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[71]  Curt Burgess,et al.  Modelling Parsing Constraints with High-dimensional Context Space , 1997 .

[72]  Padraic Monaghan,et al.  Proceedings of the 23rd annual conference of the cognitive science society , 2001 .

[73]  George A. Miller,et al.  Using a Semantic Concordance for Sense Identification , 1994, HLT.

[74]  H H Bülthoff,et al.  Psychophysical support for a two-dimensional view interpolation theory of object recognition. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[75]  J. P. Jones,et al.  The two-dimensional spatial structure of simple receptive fields in cat striate cortex. , 1987, Journal of neurophysiology.

[76]  I. Biederman.  Recognition-by-components: a theory of human image understanding. , 1987, Psychological review.

[77]  R. Schiffer.  Psychobiology of Language , 1986 .

[78]  Noam Chomsky,et al.  Modular Approaches to the Study of the Mind , 1984 .

[79]  B. Julesz.  Textons, the elements of texture perception, and their interactions , 1981, Nature.

[80]  John R. Searle,et al.  Minds, brains, and programs , 1980, Behavioral and Brain Sciences.

[81]  Saul A. Kripke,et al.  Naming and Necessity , 1980 .

[82]  J. R. Firth,et al.  Studies in Linguistic Analysis. , 1974 .

[83]  S. B. Needleman,et al.  A general method applicable to the search for similarities in the amino acid sequence of two proteins. , 1970, Journal of molecular biology.

[84]  P. Kay,et al.  Basic Color Terms: Their Universality and Evolution , 1973 .

[85]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[86]  John B. Goodenough,et al.  Contextual correlates of synonymy , 1965, CACM.

[87]  J. R. Firth,et al.  A Synopsis of Linguistic Theory, 1930-1955 , 1957 .

[88]  W. N. Locke,et al.  Machine Translation of Languages , 1956 .

[89]  Zellig S. Harris,et al.  Distributional Structure , 1954 .

[90]  C. Hartshorne,et al.  Collected Papers of Charles Sanders Peirce , 1935, Nature.

[91]  B. Russell.  II.—On Denoting , 1905 .

[92]  G. Frege.  Über Sinn und Bedeutung , 1892 .

[93]  George Kingsley Zipf,et al.  The Psychobiology of Language , 2022 .

[94]  N. Foo.  Conceptual Spaces—The Geometry of Thought , 2022 .

[95]  Christus,et al.  A General Method Applicable to the Search for Similarities in the Amino Acid Sequence of Two Proteins , 2022 .