Disrupting the Coming Robot Stampedes: Designing Resilient Information Ecologies

Machines are designed to communicate widely and efficiently. Humans, less so. We evolved social structures that function best as small subgroups interacting within larger populations. Technology changes this dynamic, by allowing all individuals to be connected at the speed of light. A dense, tightly connected population can behave like a single agent. In animals, this happens in constrained areas where stampedes can easily form. Machines do not need these kinds of conditions. The very techniques used to design best-of-breed solutions may increase the risk of dangerous mass behaviors among homogeneous machines. In this paper we argue that ecologically-based design principles such as the presence of diversity are a broadly effective strategy to defend against unintended consequences at scale.

[1]  G. A. Miller.  THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[2]  Wayne Lutters,et al.  Simon's Anthill: Mapping and Navigating Belief Spaces , 2018, ArXiv.

[3]  Felipe Cucker,et al.  Emergent Behavior in Flocks , 2007, IEEE Transactions on Automatic Control.

[4]  A. Kyle,et al.  The Flash Crash: High-Frequency Trading in an Electronic Market , 2017 .

[5]  R. Lande.  Models of speciation by sexual selection on polygenic traits. , 1981, Proceedings of the National Academy of Sciences of the United States of America.

[6]  Philip Feldman,et al.  With Maps and Mobs: Searching for Trustworthiness using Belief Spaces , 2018, CHIIR.

[7]  S. Kiesler,et al.  Group decision making and communication technology , 1992 .

[8]  S. Kauffman.  At Home in the Universe: The Search for the Laws of Self-Organization and Complexity , 1995 .

[9]  Wayne Lutters,et al.  This One Simple Trick Disrupts Digital Communities , 2018, 2018 IEEE 12th International Conference on Self-Adaptive and Self-Organizing Systems (SASO).

[10]  L. Stewart,et al.  Examining Trolls and Polarization with a Retweet Network , 2018 .

[11]  Nils J. Nilsson,et al.  A Formal Basis for the Heuristic Determination of Minimum Cost Paths , 1968, IEEE Trans. Syst. Sci. Cybern..

[12]  D. Grünbaum.  Schooling as a strategy for taxis in a noisy environment , 1998, Evolutionary Ecology.

[13]  C. S. Holling,et al.  Resilience and adaptive cycles , 2002 .