The Dynamics of Learning and the Emergence of Distributed Adaption

Abstract : This project developed fundamental theory and novel algorithms for adaptive learning in autonomous collective-agent systems. The first goal was to develop a new mathematical framework for analyzing the dynamical mechanisms that support learning in a range of novel information processing substrates. The second goal was to adapt this single-agent learning theory and associated learning algorithms to the distributed setting in which a population of autonomous agents communicate to achieve a desired group task. The results provide a mathematically sound basis for quantitatively measuring 1) the degree of individual-agent intelligence; 2) the significance of agent-environment interaction; and 3) the emergence of cooperation in agent collectives. A novel feature was that the theoretical approach synthesized recent results in the areas of machine learning, statistical inference, statistical mechanics, nonlinear dynamics, and pattern formation theory and then applied them to single-agent learning and adaptive learning in agent collectives. The central impact of the project will be its systematic and quantitative approach to predicting behavioral complexity in agents and in their environments. This will provide a foundation for measuring agent and agent-collective intelligence which, in turn, should allow for systematic engineering and monitoring of these systems.

[1]  John H. Miller,et al.  The coevolution of automata in the repeated Prisoner's Dilemma , 1996 .

[2]  Stuart J. Russell Learning agents for uncertain environments (extended abstract) , 1998, COLT' 98.

[3]  Yicheng Zhang,et al.  On the minority game: Analytical and numerical studies , 1998, cond-mat/9805084.

[4]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[5]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[6]  R. A. Brooks,et al.  Intelligence without Representation , 1991, Artif. Intell..

[7]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[8]  Paul Manneville,et al.  Dissipative Structures and Weak Turbulence , 1995 .

[9]  Edmund H. Durfee,et al.  Predicting the Expected Behavior of Agents that Learn About Agents: The CLRI Framework , 2004, Autonomous Agents and Multi-Agent Systems.

[10]  Marc M. Lankhorst Grammatical Inference with a Genetic Algorithm , 1994, EUROSIM.

[11]  E. Bonabeau,et al.  Routing in Telecommunications Networks with “ Smart ” Ant-Like Agents , 1998 .

[12]  V. Vapnik Three fundamental concepts of the capacity of learning machines , 1993 .

[13]  David Carmel,et al.  Exploration and Adaptation in Multiagent Systems: A Model-based Approach , 1997, IJCAI.

[14]  James P. Crutchfield,et al.  Evolutionary dynamics : exploring the interplay of selection, accident, neutrality, and function , 2003 .

[15]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[16]  James R. Wible,et al.  What is Complexity , 2000 .

[17]  Sean Whalen An Introduction to Computational Mechanics , 2006 .

[18]  Craig Boutilier,et al.  The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems , 1998, AAAI/IAAI.

[19]  Olof Görnerup,et al.  Objects that make objects: the population dynamics of structural complexity , 2004, Journal of The Royal Society Interface.

[20]  Rodney A. Brooks,et al.  Intelligence Without Reason , 1991, IJCAI.

[21]  J. Crutchfield The calculi of emergence: computation, dynamics and induction , 1994 .

[22]  Tad Hogg,et al.  Collective behavior of predictive agents , 1990 .

[23]  Melanie Mitchell,et al.  Finite populations induce metastability in evolutionary search , 1997 .

[24]  Pattie Maes,et al.  Kasbah: An Agent Marketplace for Buying and Selling Goods , 1996, PAAM.

[25]  Cosma Rohilla Shalizi,et al.  Symbolic Dynamics for Discrete Adaptive Games , 2002 .

[26]  A. Fisher,et al.  The Theory of critical phenomena , 1992 .

[27]  James P. Crutchfield,et al.  Reductions of Hidden Information Sources , 2004 .

[28]  Jorma Rissanen,et al.  Stochastic Complexity in Statistical Inquiry , 1989, World Scientific Series in Computer Science.

[29]  James P. Crutchfield,et al.  Hierarchical Self-Organization in the Finitary Process Soup , 2006, Artificial Life.

[30]  J. P. Crutchfield,et al.  From finite to infinite range order via annealing: the causal architecture of deformation faulting in annealed close-packed crystals , 2004 .

[31]  Michael P. Wellman,et al.  Online learning about other agents in a dynamic multiagent system , 1998, AGENTS '98.

[32]  J. Holland,et al.  Artificial Adaptive Agents in Economic Theory , 1991 .

[33]  Kristina Lisa Shalizi Pattern Discovery in Time Series, Part II: Implementation, Evaluation, and Comparison , 2022 .

[34]  K. Kaneko,et al.  How fast elements can affect slow dynamics , 2001, nlin/0108038.

[35]  James P. Crutchfield,et al.  Equations of Motion from a Data Series , 1987, Complex Syst..

[36]  Manuela M. Veloso,et al.  Multiagent Systems: A Survey from a Machine Learning Perspective , 2000, Auton. Robots.

[37]  Michael P. Wellman,et al.  Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm , 1998, ICML.

[38]  Leslie Pack Kaelbling,et al.  Acting Optimally in Partially Observable Stochastic Domains , 1994, AAAI.

[39]  H. Young,et al.  Individual Strategy and Social Structure: An Evolutionary Theory of Institutions , 1999 .

[40]  Alfred Hubler,et al.  Prediction and adaptation in an evolving chaotic environment , 1993, adap-org/9306005.

[41]  E. B. Baum,et al.  Manifesto for an evolutionary economics of intelligence , 1998 .

[42]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[43]  M Mitchell,et al.  The evolution of emergent computation. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[44]  J C Sprott,et al.  Persistent chaos in high dimensions. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[45]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[46]  Michael Kearns,et al.  Computational complexity of machine learning , 1990, ACM distinguished dissertations.

[47]  Paul Manneville,et al.  Hydrodynamics and Nonlinear Instabilities , 1998 .

[48]  J. Crutchfield,et al.  Inferring Pattern and Disorder in Close-Packed Structures from X-ray Diffraction Studies, Part II: Structure and Intrinsic Computation in Zinc Sulphide , 2003, cond-mat/0302587.

[49]  Joshua M. Epstein,et al.  Growing Artificial Societies: Social Science from the Bottom Up , 1996 .

[50]  James P. Crutchfield,et al.  Automated pattern detection - An algorithm for constructing optimally synchronizing multi-regular language filters , 2004, Theor. Comput. Sci..

[51]  J. Crutchfield,et al.  Structural information in two-dimensional patterns: entropy convergence and excess entropy. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[52]  J. Crutchfield,et al.  Upper bound on the products of particle interactions in cellular automata , 2000, nlin/0008038.

[53]  James A. Hendler,et al.  Is there an intelligent agent in your future? Nature , 1999 .

[54]  Kristina Lisa Shalizi,et al.  Pattern Discovery in Time Series, Part I: Theory, Algorithm, Analysis, and Convergence , 2002 .

[55]  James P. Crutchfield,et al.  Optimizing Epochal Evolutionary Search: Population-Size Dependent Theory , 2001, Mach. Learn..

[56]  James P. Crutchfield,et al.  Stability and diversity in collective adaptation , 2004, nlin/0408039.

[57]  D. Fudenberg,et al.  The Theory of Learning in Games , 1998 .

[58]  Michael Brereton,et al.  A Modern Course in Statistical Physics , 1981 .

[59]  T Marshall,et al.  Chaos and complexity. , 1999, The British journal of general practice : the journal of the Royal College of General Practitioners.

[60]  Victor R. Lesser,et al.  Functionally Accurate, Cooperative Distributed Systems , 1988, IEEE Transactions on Systems, Man, and Cybernetics.

[61]  C. Thompson The Statistical Mechanics of Phase Transitions , 1978 .

[62]  A. Colman,et al.  The complexity of cooperation: Agent-based models of competition and collaboration , 1998, Complex..

[63]  James P. Crutchfield,et al.  Statistical Dynamics of the Royal Road Genetic Algorithm , 1999, Theor. Comput. Sci..

[64]  James P. Crutchfield Dynamical Embodiments of Computation in Cognitive Processes , 1998 .

[65]  C. S. Wallace,et al.  An Information Measure for Classification , 1968, Comput. J..

[66]  Paul D. Scott Distributed Artificial Intelligence Meets Machine Learning: Learning in Multi-Agent Environments Edited by Gerhard Weiss , 2000, J. Artif. Soc. Soc. Simul..

[67]  Umesh V. Vazirani,et al.  An Introduction to Computational Learning Theory , 1994 .

[68]  J. Crutchfield,et al.  Coupled replicator equations for the dynamics of learning in multiagent systems. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[69]  J. Crutchfield,et al.  Regularities unseen, randomness observed: levels of entropy convergence. , 2001, Chaos.

[70]  Luca Maria Gambardella,et al.  Ant colony system: a cooperative learning approach to the traveling salesman problem , 1997, IEEE Trans. Evol. Comput..

[71]  A. Hastings Population Biology: Concepts and Models , 1996 .

[72]  Inferring Pattern and Disorder in Close-Packed Structures from X-ray Diffraction Studies, Part I: epsilon-Machine Spectral Reconstruction Theory , 2003, cond-mat/0302585.

[73]  David B. Fogel,et al.  An Evolutionary Programming Approach to Self-Adaptation on Finite State Machines , 1995, Evolutionary Programming.

[74]  J. Crutchfield,et al.  Discovering planar disorder in close-packed structures from x-ray diffraction: Beyond the fault model , 2002, cond-mat/0203290.

[75]  D. E. Matthews Evolution and the Theory of Games , 1977 .

[76]  James P. Crutchfield,et al.  Evolving One Dimensional Cellular Automata to Perform Non-trivial Collective Behavior Task: One Case Study , 2002, International Conference on Computational Science.

[77]  James P. Crutchfield,et al.  Synchronizing to the Environment: Information-Theoretic Constraints on Agent Learning , 2001, Adv. Complex Syst..

[78]  Eizo Akiyama,et al.  Chaos in learning a simple two-person game , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[79]  M. Schröder,et al.  Automatic design of multidimensional controllers by means of an evolutionary algorithm , 1997, Fuzzy Sets Syst..

[80]  Michael R. Genesereth,et al.  Software agents , 1994, CACM.

[81]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[82]  Kagan Tumer,et al.  General principles of learning-based multi-agent systems , 1999, AGENTS '99.

[83]  J. Crutchfield,et al.  Optimizing Epochal Evolutionary Search: Population-Size Dependent Theory , 1998, Machine-mediated learning.

[84]  Abraham Lempel,et al.  Compression of individual sequences via variable-rate coding , 1978, IEEE Trans. Inf. Theory.

[85]  A. Neyman Bounded complexity justifies cooperation in the finitely repeated prisoners' dilemma , 1985 .

[86]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[87]  W. Arthur,et al.  The Economy as an Evolving Complex System II , 1988 .

[88]  Takashi Ikegami,et al.  Emergence of Collective Strategies in a Prey-Predator Game Model , 1997, Artificial Life.

[89]  C. Beck,et al.  Thermodynamics of chaotic systems: Spin systems , 1993 .

[90]  M Dorigo,et al.  Ant colonies for the travelling salesman problem. , 1997, Bio Systems.

[91]  Y. Shoham,et al.  Editorial: economic principles of multi-agent systems , 1997 .

[92]  Kagan Tumer,et al.  An Introduction to Collective Intelligence , 1999, ArXiv.

[93]  W. Arthur Complexity in economic theory: inductive reasoning and bounded rationality , 1994 .

[94]  Yi-Cheng Zhang,et al.  Emergence of cooperation and organization in an evolutionary game , 1997 .

[95]  James P. Crutchfield,et al.  Information Bottlenecks, Causal States, and Statistical Relevance Bases: How to Represent Relevant Information in memoryless transduction , 2000, Adv. Complex Syst..

[96]  Young,et al.  Inferring statistical complexity. , 1989, Physical review letters.

[97]  H. Stanley,et al.  Introduction to Phase Transitions and Critical Phenomena , 1972 .

[98]  Victor R. Lesser,et al.  Coalitions Among Computationally Bounded Agents , 1997, Artif. Intell..

[99]  James P. Crutchfield,et al.  Synchronizing to Periodicity: the Transient Information and Synchronization Time of Periodic Sequences , 2002, Adv. Complex Syst..

[100]  A. Roadmapof A Roadmap of Agent Research and Development , 1995 .

[101]  K. Kaneko Dominance of Milnor attractors in globally coupled dynamical systems with more than 7+/-2 degrees of freedom. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[102]  Jorma Rissanen,et al.  Universal coding, information, prediction, and estimation , 1984, IEEE Trans. Inf. Theory.

[103]  James P. Crutchfield,et al.  Computational Mechanics: Pattern and Prediction, Structure and Simplicity , 1999, ArXiv.

[104]  J. Crutchfield,et al.  Thermodynamic depth of causal states: Objective complexity via minimal representations , 1999 .

[105]  W. Hamilton,et al.  The evolution of cooperation. , 1984, Science.

[106]  E. M.,et al.  Statistical Mechanics , 2021, Manual for Theoretical Chemistry.

[107]  Moshe Tennenholtz,et al.  Adaptive Load Balancing: A Study in Multi-Agent Learning , 1994, J. Artif. Intell. Res..

[108]  K. Sycara,et al.  This Is a Publication of the American Association for Artificial Intelligence Multiagent Systems Multiagent System Issues and Challenges Individual Agent Reasoning Task Allocation Multiagent Planning Recognizing and Resolving Conflicts Managing Communication Modeling Other Agents Managing Resources , 2022 .