Efficient compression in color naming and its evolution

Significance Semantic typology documents and explains how languages vary in their structuring of meaning. Information theory provides a formal model of communication that includes a precise definition of efficient compression. We show that color-naming systems across languages achieve near-optimal compression and that this principle explains much of the variation across languages. These findings suggest a possible process for color category evolution that synthesizes continuous and discrete aspects of previous accounts. The generality of this principle suggests that it may also apply to other semantic domains. We derive a principled information-theoretic account of cross-language semantic variation. Specifically, we argue that languages efficiently compress ideas into words by optimizing the information bottleneck (IB) trade-off between the complexity and accuracy of the lexicon. We test this proposal in the domain of color naming and show that (i) color-naming systems across languages achieve near-optimal compression; (ii) small changes in a single trade-off parameter account to a large extent for observed cross-language variation; (iii) efficient IB color-naming systems exhibit soft rather than hard category boundaries and often leave large regions of color space inconsistently named, both of which phenomena are found empirically; and (iv) these IB systems evolve through a sequence of structural phase transitions, in a single process that captures key ideas associated with different accounts of color category evolution. These results suggest that a drive for information-theoretic efficiency may shape color-naming systems across languages. This principle is not specific to color, and so it may also apply to cross-language variation in other semantic domains.

[1]  T. Florian Jaeger,et al.  Redundancy and reduction: Speakers manage syntactic information density , 2010, Cognitive Psychology.

[2]  Michael J. Berry,et al.  Predictive information in a sensory population , 2013, Proceedings of the National Academy of Sciences.

[3]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[4]  R. Saxe,et al.  A Noisy-Channel Account of Crosslinguistic Word-Order Variation , 2013, Psychological science.

[5]  Serge O Dumoulin,et al.  Measurement of population receptive fields in human early visual cortex using back-projection tomography. , 2014, Journal of vision.

[6]  Steven T Piantadosi,et al.  Word lengths are optimized for efficient communication , 2011, Proceedings of the National Academy of Sciences.

[7]  R. W. Casson,et al.  Color and Cognition in Mesoamerica: Constructing Categories as Vantages. , 1998 .

[8]  James Bailey,et al.  Information Theoretic Measures for Clusterings Comparison: Variants, Properties, Normalization and Correction for Chance , 2010, J. Mach. Learn. Res..

[9]  Naftali Tishby,et al.  Distributional Clustering of English Words , 1993, ACL.

[10]  Terry Regier,et al.  Word Meanings across Languages Support Efficient Communication , 2015 .

[11]  Thomas Hofmann,et al.  Speakers optimize information density through syntactic reduction , 2007 .

[12]  Claude E. Shannon,et al.  The mathematical theory of communication , 1950 .

[13]  P. Kay,et al.  Basic Color Terms: Their Universality and Evolution , 1973 .

[14]  K. Rose Deterministic annealing for clustering, compression, classification, regression, and related optimization problems , 1998, Proc. IEEE.

[15]  P. Kay Basic Color Terms: Their Universality and Evolution , 1969 .

[16]  Bevil R. Conway,et al.  Color naming across languages reflects color use , 2017, Proceedings of the National Academy of Sciences.

[17]  Ricard V. Solé,et al.  Least effort and the origins of scaling in human language , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[18]  P. Kay,et al.  Color naming reflects optimal partitions of color space , 2007, Proceedings of the National Academy of Sciences.

[19]  Delwin T. Lindsey,et al.  Hunter-Gatherer Color Naming Provides New Insight into the Evolution of Color Terms , 2015, Current Biology.

[20]  Imre Csiszár,et al.  Information Theory and Statistics: A Tutorial , 2004, Found. Trends Commun. Inf. Theory.

[21]  M A Nowak,et al.  Language evolution and information theory. , 2000, Journal of theoretical biology.

[22]  S. Levinson Yélî Dnye and the Theory of Basic Color Terms , 2000 .

[23]  Delwin T. Lindsey,et al.  Color naming and color consensus: “Blue” is special , 2004 .

[24]  Ohad Shamir,et al.  Learning and generalization with the information bottleneck , 2008, Theoretical Computer Science.

[25]  Angela M. Brown,et al.  The color lexicon of American English. , 2014, Journal of vision.

[26]  Ursula Hess,et al.  Handbook of Categorization in Cognitive Science , 2017 .

[27]  Noam Slonim,et al.  The Information Bottleneck : Theory and Applications , 2006 .

[28]  K. Allan,et al.  Color and Cognition in Mesoamerica: Constructing Categories as Vantages , 1999 .

[29]  Andrew M. Herbert,et al.  Facial expression recognition in a sample of neglected children , 2004 .

[30]  Aaron D. Wyner,et al.  Coding Theorems for a Discrete Source With a Fidelity CriterionInstitute of Radio Engineers, International Convention Record, vol. 7, 1959. , 1993 .

[31]  Terry Regier,et al.  Chapter 9 – THE WORLD COLOR SURVEY DATABASE , 2005 .

[32]  R. Baddeley,et al.  The Relationship between Language and the Environment Information Theory Shows Why We Have Only Three Lightness Terms , 2022 .

[33]  Naftali Tishby,et al.  The information bottleneck method , 2000, ArXiv.

[34]  R. D'Andrade,et al.  Color categories in thought and language: It's not really red, green, yellow, blue: an inquiry into perceptual color space , 1997 .

[35]  Naftali Tishby,et al.  The Information Bottleneck Revisited or How to Choose a Good Distortion Measure , 2007, 2007 IEEE International Symposium on Information Theory.