Cognitive swarming: an approach from the theoretical neuroscience of hippocampal function

The rise of mobile multi-agent robotic platforms is outpacing control paradigms for tasks that require operating in complex, realistic environments. To leverage inertial, energetic, and cost benefits of small-scale robots, critical future applications may depend on coordinating large numbers of agents with minimal onboard sensing and communication resources. In this article, we present the perspective that adaptive and resilient autonomous control of swarms of minimal agents might follow from a direct analogy with the neural circuits of spatial cognition in rodents. We focus on spatial neurons such as place cells found in the hippocampus. Two major emergent hippocampal phenomena, self-stabilizing attractor maps and temporal organization by shared oscillations, reveal theoretical solutions for decentralized self-organization and distributed communication in the brain. We consider that autonomous swarms of minimal agents with low-bandwidth communication are analogous to brain circuits of oscillatory neurons with spike-based propagation of information. The resulting notion of `neural swarm control' has the potential to be scalable, adaptive to dynamic environments, and resilient to communication failures and agent attrition. We illustrate a path toward extending this analogy into multi-agent systems applications and discuss implications for advances in decentralized swarm control.

[1]  Vijay Kumar,et al.  Estimation, Control, and Planning for Aggressive Flight With a Small Quadrotor With a Single Camera and IMU , 2017, IEEE Robotics and Automation Letters.

[2]  D. Amaral,et al.  Neurons, numbers and the hippocampal network. , 1990, Progress in brain research.

[3]  Joseph D. Monaco,et al.  Attentive Scanning Behavior Drives One-Trial Potentiation of Hippocampal Place Fields , 2014, Nature Neuroscience.

[4]  Yingxue Wang,et al.  Synchronized excitability in a network enables generation of internal neuronal sequences , 2016, eLife.

[5]  N. Daw,et al.  Hippocampal Contributions to Model-Based Planning and Spatial Memory , 2019, Neuron.

[6]  T. Hafting,et al.  Microstructure of a spatial map in the entorhinal cortex , 2005, Nature.

[7]  T. Seeley,et al.  Swarm cognition in honey bees , 2007, Behavioral Ecology and Sociobiology.

[8]  Craig W. Reynolds Flocks, herds, and schools: a distributed behavioral model , 1998 .

[9]  Olaf Sporns,et al.  A spectrum of routing strategies for brain networks , 2018, PLoS Comput. Biol..

[10]  Shantanu P. Jadhav,et al.  Interplay between Hippocampal Sharp-Wave-Ripple Events and Vicarious Trial and Error Behaviors in Decision Making , 2016, Neuron.

[11]  Brad E. Pfeiffer,et al.  Hippocampal place cell sequences depict future paths to remembered goals , 2013, Nature.

[12]  Thomas Schmickl,et al.  Introduction to WOSPP: Wave Oriented Swarm Programming Paradigm , 2018, ArXiv.

[13]  L. Frank,et al.  Neural Activity Patterns Underlying Spatial Coding in the Hippocampus. , 2018, Current topics in behavioral neurosciences.

[14]  Francesco Savelli,et al.  Origin and role of path integration in the cognitive representations of the hippocampus: computational insights into open questions , 2019, Journal of Experimental Biology.

[15]  Vijay Kumar,et al.  The grand challenges of Science Robotics , 2018, Science Robotics.

[16]  Boris S. Gutkin,et al.  Dynamics of Membrane Excitability Determine Interspike Interval Variability: A Link Between Spike Generation Mechanisms and Cortical Spike Train Statistics , 1998, Neural Computation.

[17]  R. Passingham The hippocampus as a cognitive map J. O'Keefe & L. Nadel, Oxford University Press, Oxford (1978). 570 pp., £25.00 , 1979, Neuroscience.

[18]  Ofer Tchernichovski,et al.  The dynamics of long-term exploration in the rat , 1998, Biological Cybernetics.

[19]  W. Scoville,et al.  LOSS OF RECENT MEMORY AFTER BILATERAL HIPPOCAMPAL LESIONS , 1957, Journal of neurology, neurosurgery, and psychiatry.

[20]  E. Oja Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[21]  I. Couzin Collective cognition in animal groups , 2009, Trends in Cognitive Sciences.

[22]  M Tsodyks,et al.  Attractor neural network models of spatial maps in hippocampus , 1999, Hippocampus.

[23]  H. T. Blair,et al.  Conversion of a phase‐ to a rate‐coded position signal by a three‐stage model of theta cells, grid cells, and place cells , 2008, Hippocampus.

[24]  J. Changeux,et al.  Reward-dependent learning in neuronal networks for planning and decision making. , 2000, Progress in brain research.

[25]  J. Lisman,et al.  Position reconstruction from an ensemble of hippocampal place cells: contribution of theta phase coding. , 2000, Journal of neurophysiology.

[26]  L F Abbott,et al.  Modular Realignment of Entorhinal Grid Cell Activity as a Basis for Hippocampal Remapping , 2011, The Journal of Neuroscience.

[27]  Ida Momennejad,et al.  Offline replay supports planning in human reinforcement learning , 2018, eLife.

[28]  J. Knott The organization of behavior: A neuropsychological theory , 1951 .

[29]  Libor Preucil,et al.  Swarms of micro aerial vehicles stabilized under a visual relative localization , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[30]  K. Zhang,et al.  Representation of spatial orientation by the intrinsic dynamics of the head-direction cell ensemble: a theory , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[31]  G. Buzsáki,et al.  Memory, navigation and theta rhythm in the hippocampal-entorhinal system , 2013, Nature Neuroscience.

[32]  D. Reid,et al.  Cognitive Supervisor for an Autonomous Swarm of Robots , 2017 .

[33]  C. Barry,et al.  The Role of Hippocampal Replay in Memory and Planning , 2018, Current Biology.

[34]  J. O'Keefe,et al.  The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. , 1971, Brain research.

[35]  Hyunsuk Hong,et al.  Oscillators that sync and swarm , 2017, Nature Communications.

[36]  Simon B. Laughlin,et al.  Consequences of Converting Graded to Action Potentials upon Neural Information Coding and Energy Efficiency , 2014, PLoS Comput. Biol..

[37]  Marco Dorigo,et al.  Swarm intelligence: from natural to artificial systems , 1999 .

[38]  William B. Levy,et al.  T-maze training of a recurrent CA3 model reveals the necessity of novelty-based modulation of LTP in hippocampal region CA3 , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[39]  Kechen Zhang,et al.  Universal conditions for exact path integration in neural systems , 2012, Proceedings of the National Academy of Sciences.

[40]  Kechen Zhang,et al.  Sensory Feedback, Error Correction, and Remapping in a Multiple Oscillator Model of Place-Cell Activity , 2011, Front. Comput. Neurosci..

[41]  Richard M. Murray,et al.  Recent Research in Cooperative Control of Multivehicle Systems , 2007 .

[42]  Eliseo Ferrante,et al.  Hybrid Societies: Challenges and Perspectives in the Design of Collective Behavior in Self-organizing Systems , 2016, Front. Robot. AI.

[43]  J. O’Keefe,et al.  Phase relationship between hippocampal place units and the EEG theta rhythm , 1993, Hippocampus.

[44]  J. Knierim,et al.  Attractor dynamics of spatially correlated neural activity in the limbic system. , 2012, Annual review of neuroscience.

[45]  Joseph D. Monaco,et al.  Spatial synchronization codes from coupled rate-phase neurons , 2019, PLoS Comput. Biol..

[46]  Marc Timme,et al.  Dynamic information routing in complex networks , 2015 .

[47]  Shahin Rouhani,et al.  Phase transitions and asymmetry between signal comprehension and production in biological communication , 2019, Scientific Reports.

[48]  Vijay Kumar,et al.  A Survey on Aerial Swarm Robotics , 2018, IEEE Transactions on Robotics.

[49]  W. Levy,et al.  Synapses as associative memory elements in the hippocampal formation , 1979, Brain Research.

[50]  N. Burgess Grid cells and theta as oscillatory interference: Theory and predictions , 2008, Hippocampus.

[51]  B L McNaughton,et al.  Path Integration and Cognitive Mapping in a Continuous Attractor Neural Network Model , 1997, The Journal of Neuroscience.

[52]  H. Eichenbaum Barlow versus Hebb: When is it time to abandon the notion of feature detectors and adopt the cell assembly as the unit of cognition? , 2017, Neuroscience Letters.