A mathematical theory of cooperative communication

Cooperative communication plays a central role in theories of human cognition, language, development, culture, and human-robot interaction. Prior models of cooperative communication are algorithmic in nature and do not shed light on why cooperation may yield effective belief transmission and what limitations may arise due to differences between beliefs of agents. Through a connection to the theory of optimal transport, we establishing a mathematical framework for cooperative communication. We derive prior models as special cases, statistical interpretations of belief transfer plans, and proofs of robustness and instability. Computational simulations support and elaborate our theoretical results, and demonstrate fit to human behavior. The results show that cooperative communication provably enables effective, robust belief transmission which is required to explain feats of human learning and improve human-machine interaction.

[1]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[2]  Eduardo F. Morales,et al.  An Introduction to Reinforcement Learning , 2011 .

[3]  Anca D. Dragan,et al.  Where Do You Think You're Going?: Inferring Beliefs about Dynamics from Behavior , 2018, NeurIPS.

[4]  Jamil Zaki,et al.  Affective cognition: Exploring lay theories of emotion , 2015, Cognition.

[5]  L. Kantorovich.  On the Translocation of Masses , 2006 .

[6]  Hossein Mobahi,et al.  Learning with a Wasserstein Loss , 2015, NIPS.

[7]  J. Tenenbaum,et al.  Not So Innocent , 2015, Psychological science.

[8]  I. Csiszár.  A geometric interpretation of Darroch and Ratcliff's generalized iterative scaling , 1989 .

[9]  Hyowon Gweon,et al.  Order Matters: Children's Evaluation of Underinformative Teachers Depends on Context. , 2018, Child development.

[10]  Philip A. Knight,et al.  The Sinkhorn-Knopp Algorithm: Convergence and Applications , 2008, SIAM J. Matrix Anal. Appl..

[11]  Anca D. Dragan,et al.  Pragmatic-Pedagogic Value Alignment , 2017, ISRR.

[12]  Fiery Cushman,et al.  Effectively Learning from Pedagogical Demonstrations , 2018, CogSci.

[13]  Xiaojin Zhu,et al.  Machine Teaching for Bayesian Learners in the Exponential Family , 2013, NIPS.

[14]  Marco Cuturi,et al.  Sinkhorn Distances: Lightspeed Computation of Optimal Transport , 2013, NIPS.

[15]  Jamil Zaki,et al.  Computational Models of Emotion Inference in Theory of Mind: A Review and Roadmap , 2018, Top. Cogn. Sci..

[16]  Christopher Potts,et al.  An Incremental Iterated Response Model of Pragmatics , 2018, ArXiv.

[17]  C. Villani.  Optimal Transport: Old and New , 2008 .

[18]  Patrick Shafto,et al.  Children consider prior knowledge and the cost of information both in learning from and teaching others , 2014, CogSci.

[19]  G. Dantzig.  Programming of Interdependent Activities: II Mathematical Model , 1949 .

[20]  B. Love,et al.  The myth of computational level theory and the vacuity of rational analysis , 2011, Behavioral and Brain Sciences.

[21]  Julian Jara-Ettinger,et al.  Children consider others' expected costs and rewards when deciding what to teach , 2016, CogSci.

[22]  Marco Cuturi,et al.  Computational Optimal Transport , 2019 .

[23]  Fanny Dufossé,et al.  Notes on Birkhoff-von Neumann decomposition of doubly stochastic matrices , 2016 .

[24]  Herbert H. Clark,et al.  Grounding in communication , 1991, Perspectives on socially shared cognition.

[25]  Alessandro Rudi,et al.  Differential Properties of Sinkhorn Approximation for Learning with Wasserstein Distance , 2018, NeurIPS.

[26]  Gabriel Peyré,et al.  Computational Optimal Transport , 2018, Found. Trends Mach. Learn..

[27]  U. Rothblum,et al.  Scalings of matrices which have prespecified row sums and column sums via optimization , 1989 .

[28]  G. Csibra,et al.  Natural pedagogy , 2009, Trends in Cognitive Sciences.

[29]  John R. Anderson,et al.  The Adaptive Nature of Human Categorization. , 1991 .

[30]  N. Chater,et al.  The probabilistic mind: prospects for Bayesian cognitive science , 2008 .

[31]  Hyowon Gweon,et al.  Young children consider the expected utility of others’ learning to decide what to teach , 2019, Nature Human Behaviour.

[32]  Avi Wigderson,et al.  Much Faster Algorithms for Matrix Scaling , 2017, 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS).

[33]  Joelle Pineau,et al.  Socially Adaptive Path Planning in Human Environments Using Inverse Reinforcement Learning , 2016, Int. J. Soc. Robotics.

[34]  Joshua B. Tenenbaum,et al.  Children’s understanding of the costs and rewards underlying rational action , 2015, Cognition.

[35]  Anca D. Dragan,et al.  Should Robots be Obedient? , 2017, IJCAI.

[36]  T. Koopmans.  Optimum Utilization of the Transportation System , 1949 .

[37]  H. Grice.  Logic and conversation , 1975 .

[38]  Noah D. Goodman,et al.  Knowledge and implicature: Modeling language understanding as social cognition , 2012, CogSci.

[39]  J. Tenenbaum,et al.  Theory-based Bayesian models of inductive learning and reasoning , 2006, Trends in Cognitive Sciences.

[40]  Noah D. Goodman,et al.  The double-edged sword of pedagogy: Instruction limits spontaneous exploration and discovery , 2011, Cognition.

[41]  Noah D. Goodman,et al.  A rational account of pedagogical reasoning: Teaching by, and learning from, examples , 2014, Cognitive Psychology.

[42]  Thomas L. Griffiths,et al.  Infant directed speech is consistent with teaching , 2016, Psychological review.

[43]  Martin Idel.  A review of matrix scaling and Sinkhorn's normal form for matrices and positive maps , 2016, 1609.06349.

[44]  Richard Sinkhorn,et al.  Concerning nonnegative matrices and doubly stochastic matrices , 1967 .

[45]  Anca D. Dragan,et al.  Cooperative Inverse Reinforcement Learning , 2016, NIPS.

[46]  Joshua B. Tenenbaum,et al.  Interpreting actions by attributing compositional desires , 2017, CogSci.

[47]  Zoubin Ghahramani,et al.  Probabilistic machine learning and artificial intelligence , 2015, Nature.

[48]  J. Henrich,et al.  The cultural niche: Why social learning is essential for human adaptation , 2011, Proceedings of the National Academy of Sciences.

[49]  Fiery Cushman,et al.  Showing versus doing: Teaching by demonstration , 2016, NIPS.

[50]  Jason Altschuler,et al.  Near-linear time approximation algorithms for optimal transport via Sinkhorn iteration , 2017, NIPS.

[51]  J. Tenenbaum,et al.  Children Understand That Agents Maximize Expected Utilities , 2017, Journal of experimental psychology. General.

[52]  Gabriel Peyré,et al.  Learning Generative Models with Sinkhorn Divergences , 2017, AISTATS.

[53]  Joshua B. Tenenbaum,et al.  The Naïve Utility Calculus: Computational Principles Underlying Commonsense Psychology , 2016, Trends in Cognitive Sciences.

[54]  Ines Gloeckner,et al.  Relevance Communication And Cognition , 2016 .

[55]  Joshua B. Tenenbaum,et al.  Beliefs about desires: Children's understanding of how knowledge and preference influence choice , 2015, CogSci.

[56]  Michael C. Frank,et al.  Learning From Others , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[57]  Nicolas Courty,et al.  Optimal Transport for Domain Adaptation , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  Xiaojin Zhu,et al.  Machine Teaching: An Inverse Problem to Machine Learning and an Approach Toward Optimal Education , 2015, AAAI.

[59]  J. Darroch,et al.  Generalized Iterative Scaling for Log-Linear Models , 1972 .

[60]  R. Duncan Luce,et al.  Individual Choice Behavior: A Theoretical Analysis , 1979 .

[61]  S. Levinson.  Presumptive Meanings: The theory of generalized conversational implicature , 2001 .

[62]  M. Tomasello.  The Cultural Origins of Human Cognition , 2000 .

[63]  Michael Franke,et al.  Emerging abstractions: Lexical conventions are shaped by communicative context , 2018, CogSci.

[64]  E. Davis,et al.  How Robust Are Probabilistic Models of Higher-Level Cognition? , 2013, Psychological science.

[65]  Hans Schneider,et al.  The spectrum of a nonlinear operator associated with a matrix , 1969 .

[66]  Charles Kemp,et al.  How to Grow a Mind: Statistics, Structure, and Abstraction , 2011, Science.

[67]  Christopher G. Lucas,et al.  People learn other people’s preferences through inverse decision-making , 2017, Cognition.

[68]  Chris L. Baker,et al.  Rational quantitative attribution of beliefs, desires and percepts in human mentalizing , 2017, Nature Human Behaviour.

[69]  J. Lorenz,et al.  On the scaling of multidimensional matrices , 1989 .

[70]  Scott Cheng-Hsin Yang,et al.  Optimal Cooperative Inference , 2018, AISTATS.

[71]  I. Csiszár.  $I$-Divergence Geometry of Probability Distributions and Minimization Problems , 1975 .

[72]  Noah D. Goodman,et al.  The mentalistic basis of core social cognition: experiments in preverbal infants and a computational model. , 2013, Developmental science.

[73]  Patrick Shafto,et al.  Generalizing the theory of cooperative inference , 2019, AISTATS.

[74]  Joshua B. Tenenbaum,et al.  Ten-month-old infants infer the value of goals from the costs of actions , 2017, Science.

[75]  Anca D. Dragan,et al.  Literal or Pedagogic Human? Analyzing Human Model Misspecification in Objective Learning , 2019, UAI.

[76]  Patrick Shafto,et al.  Toward a general, scaleable framework for Bayesian teaching with applications to topic models , 2016, ArXiv.

[77]  S. Fienberg.  An Iterative Procedure for Estimation in Contingency Tables , 1970 .

[78]  Julian Jara-Ettinger,et al.  Theory of mind as inverse reinforcement learning , 2019, Current Opinion in Behavioral Sciences.

[79]  Elizabeth Gilbert,et al.  Reproducibility Project: Results (Part of symposium called "The Reproducibility Project: Estimating the Reproducibility of Psychological Science") , 2014 .

[80]  Noah D. Goodman,et al.  Animal, dog, or dalmatian? Level of abstraction in nominal referring expressions , 2016, CogSci.

[81]  Michael Franke,et al.  Probabilistic pragmatics, or why Bayes’ rule is probably important for pragmatics , 2016 .

[82]  Siddhartha S. Srinivasa,et al.  Legibility and predictability of robot motion , 2013, 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[83]  Chris L. Baker,et al.  Action understanding as inverse planning , 2009, Cognition.

[84]  P. J. Chase,et al.  Order independence and factor convergence in iterative scaling , 1993 .

[85]  Michael C. Frank,et al.  Review Pragmatic Language Interpretation as Probabilistic Inference , 2022 .

[86]  Y. Brenier.  Polar Factorization and Monotone Rearrangement of Vector-Valued Functions , 1991 .

[87]  J. Tenenbaum,et al.  Structure and strength in causal induction , 2005, Cognitive Psychology.

[88]  Noah D. Goodman,et al.  Teaching Games : Statistical Sampling Assumptions for Learning in Pedagogical Situations , 2008 .

[89]  D. Sperber,et al.  Relevance: Communication and Cognition , 1989 .