The Large-Scale Structure of Semantic Networks: Statistical Analyses and a Model of Semantic Growth

We present statistical analyses of the large-scale structure of 3 types of semantic networks: word associations, WordNet, and Roget's Thesaurus. We show that they have a small-world structure, characterized by sparse connectivity, short average path lengths between words, and strong local clustering. In addition, the distributions of the number of connections follow power laws that indicate a scale-free pattern of connectivity, with most nodes having relatively few connections joined together through a small number of hubs with many connections. These regularities have also been found in certain other complex natural networks, such as the World Wide Web, but they are not consistent with many conventional models of semantic organization, based on inheritance hierarchies, arbitrarily structured networks, or high-dimensional vector spaces. We propose that these structures reflect the mechanisms by which semantic networks grow. We describe a simple model for semantic growth, in which each new word or concept is connected to an existing network by differentiating the connectivity pattern of an existing node. This model generates appropriate small-world statistics and power-law connectivity distributions, and it also suggests one possible mechanistic basis for the effects of learning history variables (age of acquisition, usage frequency) on behavioral performance in semantic processing tasks.

[1]  Steven A. Sloman,et al.  Feature Centrality and Conceptual Coherence , 1998, Cogn. Sci..

[2]  De Vries.  Book review: R.C. O'Reilly and Y. Munakata: Computational explorations in cognitive neuroscience: understanding the mind by stimulating the brain. Cambridge, Mass: The MIT Press. , 2002 .

[3]  R. O’Reilly,et al.  Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain , 2000 .

[4]  Hinrich Schütze,et al.  Book Reviews: Foundations of Statistical Natural Language Processing , 1999, CL.

[5]  M. A. Griffin,et al.  Information Processing Systems , 1976 .

[6]  J. W. Hutchinson,et al.  Nearest neighbor analysis of psychological spaces. , 1986 .

[7]  S. Sloman.  Categorical Inference Is Not a Tree: The Myth of Inheritance Hierarchies , 1998, Cognitive Psychology.

[8]  Routledge,et al.  Routledge Encyclopedia of Philosophy , 1998 .

[9]  Ned Block,et al.  Semantics, Conceptual Role , 1997 .

[10]  A. McEleney.  Organization and Emergence of Semantic Knowledge: A Parallel-Distributed Processing Approach , 2005 .

[11]  J. Hodges,et al.  Charting the progression in semantic dementia: implications for the organisation of semantic memory. , 1995 .

[12]  J. Fodor,et al.  Concepts: Where Cognitive Science Went Wrong , 1998 .

[13]  Dedre Gentner,et al.  Some interesting differences between nouns and verbs , 1981 .

[14]  J. Deese.  The structure of associations in language and thought , 1966 .

[15]  Allan Collins,et al.  A spreading-activation theory of semantic processing , 1975 .

[16]  Andrew W. Ellis,et al.  Age of Acquisition Norms for a Large Set of Object Names and Their Relation to Adult Estimates and Other Variables , 1997 .

[17]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[18]  Lance J. Rips,et al.  Semantic distance and the verification of semantic relations , 1973 .

[19]  H. Kucera,et al.  Computational analysis of present-day American English , 1967 .

[20]  A. Tversky.  Features of Similarity , 1977 .

[21]  Martin Davies,et al.  Routledge Encyclopedia of Philosophy Online , 2000 .

[22]  Lada A. Adamic.  The Small World Web , 1999, ECDL.

[23]  R. Logie,et al.  Age-of-acquisition, imagery, concreteness, familiarity, and ambiguity measures for 1,944 words , 1980 .

[24]  Jon M. Kleinberg,et al.  The Web as a Graph: Measurements, Models, and Methods , 1999, COCOON.

[25]  James L. McClelland,et al.  Understanding normal and impaired word reading: computational principles in quasi-regular domains. , 1996, Psychological review.

[26]  Fred Sommers.  Structural ontology , 1971 .

[27]  S. Strogatz.  Exploring complex networks , 2001, Nature.

[28]  George Kingsley Zipf,et al.  Human behavior and the principle of least effort , 1949 .

[29]  S. Carey.  The child as word learner , 1978 .

[30]  T. Mattfeldt.  Stochastic Geometry and Its Applications , 1996 .

[31]  George A. Miller,et al.  Introduction to WordNet: An On-line Lexical Database , 1990 .

[32]  Peter W. Foltz,et al.  An introduction to latent semantic analysis , 1998 .

[33]  G. Miller,et al.  Cognitive science. , 1981, Science.

[34]  John R. Anderson,et al.  Learning and Memory: An Integrated Approach , 1994 .

[35]  John R. Anderson.  Learning and memory: An integrated approach, 2nd ed. , 2000 .

[36]  J. Hoffmann,et al.  The European Society for Cognitive Psychology , 1999 .

[37]  Jon M. Kleinberg,et al.  Small-World Phenomena and the Dynamics of Information , 2001, NIPS.

[38]  H E Stanley,et al.  Classes of small-world networks. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[39]  R. Brown.  How shall a thing be called. , 1958, Psychological review.

[40]  A W Ellis,et al.  Contrasting effects of age of acquisition and word frequency on auditory and visual lexical decision , 1998, Memory & cognition.

[41]  A. Ellis,et al.  Last in, First to Go: Age of Acquisition and Naming in the Elderly , 1998, Brain and Language.

[42]  Albert-László Barabási,et al.  Statistical mechanics of complex networks , 2001, ArXiv.

[43]  N. Gee,et al.  Interpreting the influence of implicitly activated memories on recall and recognition. , 1998, Psychological review.

[44]  D. Slobin,et al.  Studies of child language development , 1973 .

[45]  Mark Steyvers,et al.  Finding scientific topics , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[46]  George W. Davidson,et al.  Roget's Thesaurus of English Words and Phrases , 1982 .

[47]  P. Erdos,et al.  On the evolution of random graphs , 1984 .

[48]  Noam Chomsky.  Review of B.F. Skinner, Verbal Behavior , 1959 .

[49]  Albert-László Barabási,et al.  Error and attack tolerance of complex networks , 2000, Nature.

[50]  M. Ross Quillian,et al.  Retrieval time from semantic memory , 1969 .

[51]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[52]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[53]  G. Miller,et al.  Linguistic theory and psychological reality , 1982 .

[54]  Sharon L. Milgram,et al.  The Small World Problem , 1967 .

[55]  B. Skinner.  The distribution of associated words , 1937 .

[56]  H. Simon,et al.  ON A CLASS OF SKEW DISTRIBUTION FUNCTIONS , 1955 .

[57]  B. Bollobás.  The evolution of random graphs , 1984 .

[58]  Albert-László Barabási,et al.  Internet: Diameter of the World-Wide Web , 1999, Nature.

[59]  Lada A. Adamic.  Zipf, Power-laws, and Pareto-a ranking tutorial , 2000 .

[60]  Beth Levin,et al.  English Verb Classes and Alternations: A Preliminary Investigation , 1993 .

[61]  Garrison W. Cottrell,et al.  The Early Word Catches the Weights , 2000, NIPS.

[62]  J. Carroll,et al.  Word Frequency and Age of Acquisition as Determiners of Picture-Naming Latency , 1973 .

[63]  M. Newman,et al.  The structure of scientific collaboration networks. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[64]  Massimo Marchiori,et al.  Error and attacktolerance of complex network s , 2004 .

[65]  Noam Chomsky,et al.  A Review of B. F. Skinner's Verbal Behavior , 1980 .

[66]  Matthew A. Lambon Ralph,et al.  Naming in semantic dementia—what matters? , 1998, Neuropsychologia.

[67]  D. Watts,et al.  Small Worlds: The Dynamics of Networks between Order and Randomness , 2001 .

[68]  M. Alexander,et al.  Principles of Neural Science , 1981 .

[69]  Eve V. Clark,et al.  The Lexicon in Acquisition , 1996 .

[70]  A. Rbnyi.  ON THE EVOLUTION OF RANDOM GRAPHS , 2001 .

[71]  Thomas L. Griffiths,et al.  A probabilistic approach to semantic representation , 2019, Proceedings of the Twenty-Fourth Annual Conference of the Cognitive Science Society.

[72]  Mark Steyvers,et al.  Small Worlds in Semantic Networks , 2022 .

[73]  Roger W. Brown,et al.  Words and Things. , 1959 .

[74]  S. Pinker.  The Language Instinct , 1994 .

[75]  Michael Gasser,et al.  Learning Nouns and Adjectives: A Connectionist Account , 1998 .

[76]  W. Strange.  Evolution of language. , 1984, JAMA.

[77]  Michael B. Lewis,et al.  Re-evaluating age-of-acquisition effects: are they simply cumulative-frequency effects? , 2001, Cognition.

[78]  R. Albert,et al.  The large-scale organization of metabolic networks , 2000, Nature.

[79]  Gabriel Furmuzachi,et al.  WORDS AND THINGS , 1906, British medical journal.

[80]  M. L. Lambon Ralph,et al.  Age of acquisition effects in adult lexical processing reflect loss of plasticity in maturing systems: insights from connectionist networks. , 2000, Journal of experimental psychology. Learning, memory, and cognition.

[81]  A. Dickson.  On Evolution , 1884, Science.

[82]  Alan M. Frieze,et al.  Random graphs , 2006, SODA '06.

[83]  E. Warrington.  Quarterly Journal of Experimental Psychology the Selective Impairment of Semantic Memory the Selective Impairment of Semantic Memory , 2022 .

[84]  J. Macnamara.  Names for Things: A Study in Human Learning , 1984 .

[85]  Jie Wu,et al.  Small Worlds: The Dynamics of Networks between Order and Randomness , 2003 .

[86]  Charles E. Caton,et al.  Semantic and Conceptual Development: An Ontological Perspective , 1982 .

[87]  Partha Niyogi,et al.  Evolutionary Consequences of Language Learning , 1997 .

[88]  M Brysbaert,et al.  Age-of-acquisition effects in semantic processing tasks. , 2000, Acta psychologica.

[89]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[90]  Sergey Brin,et al.  The Anatomy of a Large-Scale Hypertextual Web Search Engine , 1998, Comput. Networks.