Distributed Adaptive Control: A theory of the Mind, Brain, Body Nexus

Abstract Distributed Adaptive Control (DAC) is a theory of the design principles underlying the Mind, Brain, Body Nexus (MBBN) that has been developed over the last 20 years. DAC assumes that the brain maintains stability between an embodied agent and its environment through action. It postulates that in order to act, or know how, the brain has to answer four fundamental questions: why, what, where, when. Thus the function of the brain is to continuously solve the, so called, H4W problem. The DAC theory is expressed as a robot based neural architecture organized in two complementary structures: layers and columns. The organizational layers are called: reactive, adaptive and contextual and its columnar organization defines the processing of states of the world, the self and the generation of action. Each layer is described with respect to its key hypotheses, implementation and specific benchmarks. After this overview of the key elements of DAC, the mapping of its key assumptions towards the invertebrate and mammalian brain is described. In particular, this review will focus on the systems involved in realizing the core principles underlying the reactive layer: the allostatic control of fundamental behavior systems in the vertebrate brain and the emergent non-linearity through neuronal mass action in the locust brain. The adaptive layer will be analyzed in terms of the classical conditioning paradigm and its neuronal substrate the amygdala-cerebellum-neocortex complex together with episodic memory and the formation of sense-act couplets in the hippocampus. For the contextual layer the ability of circuits in the prefrontal cortex to acquire and express contextual plans for action is described. The general overview of DAC’s explanation of MBBN is combined by examples of application scenarios in which DAC has been validated including mobile and humanoid robots, neurorehabilitation and the large-scale interactive space Ada. After 20 years of research DAC can be considered a mature theory of MBBN. It has build up a track record of explaining core aspects of mind, brain and behavior, has made testable and verified predictions at the level of behavior, physiology and anatomy, has been shown to be able to control complex real-world artefacts and has been successfully applied to brain repair and neurorehabilitation. Currently DAC is extended to capture the phenomenon of consciousness, the ultimate challenge in the study of the Mind, Brain, Body Nexus.

[1]  Paul F. M. J. Verschure,et al.  An integrated computational model of the two phase theory of classical conditioning , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[2]  J. Gray The neuropsychology of anxiety. , 1985, Issues in mental health nursing.

[3]  John R. Anderson Cognitive Psychology and Its Implications , 1980 .

[4]  P. Glimcher Indeterminacy in brain and behavior. , 2005, Annual review of psychology.

[5]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[6]  K. H. Stauder,et al.  Psychology of the Child , 1959 .

[7]  Paul F. M. J. Verschure,et al.  Integrating neuroscience-based models towards an autonomous biomimetic Synthetic Forager , 2011, 2011 IEEE International Conference on Robotics and Biomimetics.

[8]  R. Dantzer The Psychology of Fear and Stress, J.A. Gray (Ed.). Cambridge University Press, Cambridge (1987), viii and 422 pp, ISBN 0-521-27098-7 , 1989 .

[9]  John L Kubie,et al.  Heading‐vector navigation based on head‐direction cells and path integration , 2009, Hippocampus.

[10]  Giulio Sandini,et al.  The iCub humanoid robot: an open platform for research in embodied cognition , 2008, PerMIS.

[11]  H. Markram The Blue Brain Project , 2006, Nature Reviews Neuroscience.

[12]  Paul F. M. J. Verschure,et al.  Neuron Report TheMechanismofRateRemappingintheDentateGyrus , 2010 .

[13]  Paul F. M. J. Verschure,et al.  Learning from the Moth: A Comparative Study of Robot‐Based Odor Source Localization Strategies , 2009 .

[14]  Paul F. M. J. Verschure,et al.  Local and Global Gating of Synaptic Plasticity , 2000, Neural Computation.

[15]  Rodney A. Brooks,et al.  A Robust Layered Control Syste For A Mobile Robot , 2022 .

[16]  Calvin S. Hall,et al.  What People Dream About , 1951 .

[17]  T. Robbins,et al.  The hippocampal–striatal axis in learning, prediction and goal-directed behavior , 2011, Trends in Neurosciences.

[18]  M. Kilgard,et al.  Cortical map reorganization enabled by nucleus basalis activity. , 1998, Science.

[19]  J. Konorski Conditioned reflexes and neuron organization. , 1948 .

[20]  H. Reichert,et al.  Evolution of Nervous Systems , 2007 .

[21]  Angelo Cangelosi,et al.  Grounding language in action and perception: from cognitive agents to humanoid robots. , 2010, Physics of life reviews.

[22]  P. König,et al.  A Model of the Ventral Visual System Based on Temporal Stability and Local Memory , 2006, PLoS biology.

[23]  Massimo Vergassola,et al.  ‘Infotaxis’ as a strategy for searching without gradients , 2007, Nature.

[24]  Peter Ford Dominey,et al.  Anticipation and initiative in human-humanoid interaction , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.

[25]  J. Lisman,et al.  Role of the dual entorhinal inputs to hippocampus: a hypothesis based on cue/action (non-self/self) couplets. , 2007, Progress in brain research.

[26]  E. Oja Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[27]  W. Schultz Behavioral theories and the neurophysiology of reward. , 2006, Annual review of psychology.

[28]  José M. R. Delgado,et al.  Integrative Activity of the Brain , 1968, The Yale Journal of Biology and Medicine.

[29]  Paul F. M. J. Verschure,et al.  Allostatic control for robot behaviour regulation: An extension to path planning , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[30]  Paul F. M. J. Verschure,et al.  Decoding a Temporal Population Code , 2004, Neural Computation.

[31]  C. Espinosa-Garcia,et al.  Neuroprotective effects of progesterone and allopregnanolone on long-term cognitive outcome after global cerebral ischemia. , 2011, Restorative neurology and neuroscience.

[32]  Aurika Richkiene,et al.  The Global and the Local : The History of Science and the Cultural Integration of Europe , 2008 .

[33]  Dhanistha Panyasak,et al.  Circuits , 1995, Annals of the New York Academy of Sciences.

[34]  Tobi Delbrück,et al.  Ada - intelligent space: an artificial creature for the SwissExpo.02 , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[35]  Paul F. M. J. Verschure,et al.  A fly-locust based neuronal control system applied to an unmanned aerial vehicle: the invertebrate neuronal principles for course stabilization, altitude control and collision avoidance , 2007, Int. J. Robotics Res..

[36]  Paul F. M. J. Verschure,et al.  Unifying perceptual and behavioral learning with a correlative subspace learning rule , 2010, Neurocomputing.

[37]  Uğur M Erdem,et al.  A goal‐directed spatial navigation model using forward trajectory planning based on grid cells , 2012, The European journal of neuroscience.

[38]  P. Verschure,et al.  A biologically based model for the integration of sensory–motor contingencies in rules and plans: A prefrontal cortex based extension of the Distributed Adaptive Control architecture , 2011, Brain Research Bulletin.

[39]  David J. Foster,et al.  Reverse replay of behavioural sequences in hippocampal place cells during the awake state , 2006, Nature.

[40]  E. Miller,et al.  An integrative theory of prefrontal cortex function. , 2001, Annual review of neuroscience.

[41]  P. Verschure,et al.  The cerebellum in action: a simulation and robotics study , 2002, The European journal of neuroscience.

[42]  G. Tononi Consciousness as Integrated Information: a Provisional Manifesto , 2008, The Biological Bulletin.

[43]  Rüdiger Dillmann,et al.  Robot Companions for Citizens , 2011, FET.

[44]  J. Lisman The theta/gamma discrete phase code occuring during the hippocampal phase precession may be a more general brain coding scheme , 2005, Hippocampus.

[45]  Tobi Delbrück,et al.  A tactile luminous floor for an interactive autonomous space , 2007, Robotics Auton. Syst..

[46]  Edward F. Moore,et al.  Gedanken-Experiments on Sequential Machines , 1956 .

[47]  G. Edelman,et al.  Synthetic neural modeling applied to a real-world artifact. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[48]  N. Spruston,et al.  Action potential initiation and backpropagation in neurons of the mammalian CNS , 1997, Trends in Neurosciences.

[49]  Paul F. M. J. Verschure,et al.  A Model of Grid Cells Based on a Twisted Torus Topology , 2007, Int. J. Neural Syst..

[50]  P. Redgrave,et al.  The basal ganglia: a vertebrate solution to the selection problem? , 1999, Neuroscience.

[51]  Paul F. M. J. Verschure,et al.  An insect-based method for learning landmark reliability using expectation reinforcement in dynamic environments , 2010, 2010 IEEE International Conference on Robotics and Automation.

[52]  D C Blanchard,et al.  Antipredator defensive behaviors in a visible burrow system. , 1989, Journal of comparative psychology.

[53]  Karl J. Friston The free-energy principle: a unified brain theory? , 2010, Nature Reviews Neuroscience.

[54]  M. Moser,et al.  Pattern Separation in the Dentate Gyrus and CA3 of the Hippocampus , 2007, Science.

[55]  F. Barth,et al.  Sensors and Sensing in Biology and Engineering , 2003, Springer Vienna.

[56]  A. Redish Beyond the Cognitive Map: From Place Cells to Episodic Memory , 1999 .

[57]  Verschure Paul The variance of neuronal activity in dorsal premotor area is predictive of behavioral performance: The influence of trial history in decision making , 2012 .

[58]  Giacomo Indiveri,et al.  The Cerebellum Chip: an Analog VLSI Implementation of a Cerebellar Model of Classical Conditioning , 2004, NIPS.

[59]  L. S. Kogan Review of Principles of Behavior. , 1943 .

[60]  Pat Langley,et al.  Cognitive Architectures and General Intelligent Systems , 2006, AI Mag..

[61]  Ben J. A. Kröse,et al.  Distributed adaptive control: The self-organization of structured behavior , 1992, Robotics Auton. Syst..

[62]  Dominic W. Massaro,et al.  SPEECH RECOGNITION AND SENSORY INTEGRATION , 1998 .

[63]  P. Verschure Smolensky's theory of mind , 1990, Behavioral and Brain Sciences.

[64]  V. Braitenberg Vehicles, Experiments in Synthetic Psychology , 1984 .

[65]  P. Verschure Taking connectionism seriously: the vague promise of subsymbolism and an alternative , 1992 .

[66]  Sebastian Thrun,et al.  Monte Carlo POMDPs , 1999, NIPS.

[67]  Paul F. M. J. Verschure,et al.  What Can Robots Tell Us About Brains? A Synthetic Approach Towards the Study of Learning and Problem Solving , 1999, Reviews in the neurosciences.

[68]  Johannes J. Letzkus,et al.  A disinhibitory microcircuit for associative fear learning in the auditory cortex , 2011, Nature.

[69]  Boris I. Shraiman,et al.  Olfactory search at high Reynolds number , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[70]  N. Weinberger Specific long-term memory traces in primary auditory cortex , 2004, Nature Reviews Neuroscience.

[71]  A. J. Bell,et al.  Levels and loops: the future of artificial intelligence and neuroscience. , 1999, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[72]  Paul F. M. J. Verschure,et al.  Categorization, representations, and the dynamics of system-environment interaction: a case study in autonomous systems , 1993 .

[73]  E. Tolman Prediction of vicarious trial and error by means of the schematic sowbug. , 1939 .

[74]  Ronald C. Arkin,et al.  An Behavior-based Robotics , 1998 .

[75]  J. H. Curtis,et al.  Learning Theory and Behavior , 1960 .

[76]  Rajesh P. N. Rao,et al.  Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. , 1999 .

[77]  David E J Linden,et al.  Who's in charge? Free will and the science of the brain , 2014, Cognitive neuropsychiatry.

[78]  Paul F. M. J. Verschure,et al.  A real-world rational agent: unifying old and new AI , 2003, Cogn. Sci..

[79]  Paul F. M. J. Verschure,et al.  Neuroscience, virtual reality and neurorehabilitation: Brain repair as a validation of brain theory , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[80]  E. A. Arbas,et al.  Evolution in nervous systems. , 1991, Annual review of neuroscience.

[81]  D. Massaro Perceiving talking faces: from speech perception to a behavioral principle , 1999 .

[82]  Stephen Grossberg,et al.  Competitive Learning: From Interactive Activation to Adaptive Resonance , 1987, Cogn. Sci..

[83]  Florentin Wörgötter,et al.  Strongly Improved Stability and Faster Convergence of Temporal Sequence Learning by Using Input Correlations Only , 2006, Neural Computation.

[84]  Jing Gu,et al.  An artificial moth: Chemical source localization using a robot based neuronal model of moth optomotor anemotactic search , 2006, Auton. Robots.

[85]  Paul F. M. J. Verschure,et al.  The fusion of multiple sources of information in the organization of goal-oriented behavior: Spatial attention versus integration , 2007 .

[86]  Peter König,et al.  Invariant representations of visual patterns in a temporal population code , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[87]  F. Mondada,et al.  Modeling system-environment interaction: the complimentary roles of simulation and real world artifacts , 1993 .

[88]  P. Verschure,et al.  Virtual reality based rehabilitation speeds up functional recovery of the upper extremities after stroke: a randomized controlled pilot study in the acute phase of stroke using the rehabilitation gaming system. , 2011, Restorative neurology and neuroscience.

[89]  Paul F. M. J. Verschure,et al.  Non-Linear Neuronal Responses as an Emergent Property of Afferent Networks: A Case Study of the Locust Lobula Giant Movement Detector , 2010, PLoS Comput. Biol..

[90]  W. Walter A Machine that Learns , 1951 .

[91]  B. Sakmann,et al.  Active propagation of somatic action potentials into neocortical pyramidal cell dendrites , 1994, Nature.

[92]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[93]  Richard Reviewer-Granger Unified Theories of Cognition , 1991, Journal of Cognitive Neuroscience.

[94]  Mark D. Humphries,et al.  A robot model of the basal ganglia: Behavior and intrinsic processing , 2006, Neural Networks.

[95]  Xiao-Jing Wang Decision Making in Recurrent Neuronal Circuits , 2008, Neuron.

[96]  J. Fuster Prefrontal Cortex , 2018 .

[97]  Paul F. M. J. Verschure,et al.  The fusion of multiple sources of information in the organization of goal-oriented behavior , 2009 .

[98]  Paul F. M. J. Verschure,et al.  Insect-Like mapless navigation based on head direction cells and contextual learning using chemo-visual sensors , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[99]  H. Roitblat,et al.  Categorization , Representations , and the Dynamics of System-Environment Interaction : a case study in autonomous systems , 1992 .

[100]  S. Coste,et al.  Allostasis, Homeostasis, and the Costs of Physiological Adaptation , 2006 .

[101]  Richard F. Thompson In search of memory traces. , 2005, Annual review of psychology.

[102]  Timothy D. Hanks,et al.  Probabilistic Population Codes for Bayesian Decision Making , 2008, Neuron.

[103]  Paul F. M. J. Verschure,et al.  Synthetic epistemology: the acquisition, retention, and expression of knowledge in natural and synthetic systems , 1998, 1998 IEEE International Conference on Fuzzy Systems Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36228).

[104]  Márcio O. Costa,et al.  Design for a Brain Revisited: The Neuromorphic Design and Functionality of the Interactive Space 'Ada' , 2003, Reviews in the neurosciences.

[105]  Peter König,et al.  Learning sensory maps with real-world stimuli in real time using a biophysically realistic learning rule , 2002, IEEE Trans. Neural Networks.

[106]  Paul F. M. J. Verschure,et al.  Allostatic Control for Robot Behavior Regulation: a Comparative Rodent-Robot Study , 2010, Adv. Complex Syst..

[107]  Philip W. Anderson,et al.  WHAT MAD PURSUIT. A Personal View of Scientific Discovery. , 1989 .

[108]  Paul F. M. J. Verschure,et al.  A wavelet-based neural model to optimize and read out a temporal population code , 2012, Front. Comput. Neurosci..

[109]  Paul F. M. J. Verschure,et al.  The neuronal substrate underlying order and interval representations in sequential tasks: A biologically based robot study , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[110]  Paul F. M. J. Verschure,et al.  An interactive space that learns to influence human behavior , 2005, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[111]  Paul F M J Verschure,et al.  Prediction of the position of an animal based on populations of grid and place cells: a comparative simulation study. , 2007, Journal of integrative neuroscience.

[112]  P. Verschure,et al.  Adaptive fields: distributed representations of classically conditioned associations , 1991 .

[113]  Paul F. M. J. Verschure,et al.  The encoding of complex visual stimuli by a canonical model of the primary visual cortex: Temporal population code for face recognition on the iCub robot , 2011, 2011 IEEE International Conference on Robotics and Biomimetics.

[114]  J R Novell From Da Vinci to Harvey: The Development of Mechanical Analogy in Medicine from 1500 to 1650 , 1990, Journal of the Royal Society of Medicine.

[115]  D. Barraclough,et al.  Prefrontal cortex and decision making in a mixed-strategy game , 2004, Nature Neuroscience.

[116]  John R. Searle,et al.  Minds, brains, and programs , 1980, Behavioral and Brain Sciences.

[117]  Paul F. M. J. Verschure,et al.  A bottom up approach towards the acquisition and expression of sequential representations applied to a behaving real-world device: Distributed Adaptive Control III , 1998, Neural Networks.

[118]  I. Pavlov,et al.  Conditioned reflexes: An investigation of the physiological activity of the cerebral cortex , 2010, Annals of Neurosciences.

[119]  K. Doya Modulators of decision making , 2008, Nature Neuroscience.

[120]  Eric R. Kandel,et al.  In Search of Memory , 2006 .

[121]  Joaquín M. Fuster,et al.  Cortex and Memory: Emergence of a New Paradigm , 2009, Journal of Cognitive Neuroscience.

[122]  Sebastian Thrun,et al.  Probabilistic robotics , 2002, CACM.

[123]  Pat Langley,et al.  Cognitive architectures: Research issues and challenges , 2009, Cognitive Systems Research.

[124]  Oussama Khatib,et al.  Real-Time Obstacle Avoidance for Manipulators and Mobile Robots , 1985, Autonomous Robot Vehicles.

[125]  A. Borst,et al.  Fly motion vision. , 2010, Annual review of neuroscience.

[126]  Paul F. M. J. Verschure,et al.  PASAR: An integrated model of prediction, anticipation, sensation, attention and response for artificial sensorimotor systems , 2012, Inf. Sci..

[127]  P. Sterling,et al.  Allostasis: A new paradigm to explain arousal pathology. , 1988 .

[128]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[129]  Paul F. M. J. Verschure,et al.  Environmentally mediated synergy between perception and behaviour in mobile robots , 2003, Nature.

[130]  P. Lazarsfeld Principles of Topological Psychology , 1938 .

[131]  Bill S Hansson,et al.  A bug's smell – research into insect olfaction , 2002, Trends in Neurosciences.

[132]  Paul F. M. J. Verschure,et al.  Locust’s Looming Detectors for Robot Sensors , 2003 .

[133]  R. F. Tredgold,et al.  The Living Brain , 1954, Mental Health.

[134]  Paul F. M. J. Verschure,et al.  A Biologically Based Chemo-Sensing UAV for Humanitarian Demining , 2007 .

[135]  Paul F. M. J. Verschure,et al.  Formal minds and biological brains: AI and Edelman's extended theory of neuronal group selection , 1993, IEEE Expert.

[136]  M. Carandini From circuits to behavior: a bridge too far? , 2012, Nature Neuroscience.

[137]  Wlodzislaw Duch,et al.  Cognitive Architectures: Where do we go from here? , 2008, AGI.

[138]  Paul F. M. J. Verschure,et al.  Live Soundscape Composition Based on Synthetic Emotions , 2003, IEEE Multim..

[139]  Rodney J. Douglas,et al.  Distributed Adaptive Control 5: Bayesian Theory of Decision Making, Implemented on Simulated and Real Robots , 2001 .

[140]  B. Skinner,et al.  Principles of Behavior , 1944 .

[141]  Randall D. Beer,et al.  The brain has a body: adaptive behavior emerges from interactions of nervous system, body and environment , 1997, Trends in Neurosciences.

[142]  J. Russell A circumplex model of affect. , 1980 .

[143]  G D ELLISON,et al.  Separation of the Salivary and Motor Responses in Instrumental Conditioning , 1964, Science.

[144]  L. Steels The symbol grounding problem has been solved, so what’s next? , 2008 .

[145]  Albert K. Lee,et al.  Memory of Sequential Experience in the Hippocampus during Slow Wave Sleep , 2002, Neuron.

[146]  N. Mackintosh The psychology of animal learning , 1974 .

[147]  J. Stevens,et al.  Animal Intelligence , 1883, Nature.

[148]  Holger G. Krapp,et al.  Multiplication and stimulus invariance in a looming-sensitive neuron , 2004, Journal of Physiology-Paris.

[149]  Rolf Pfeifer,et al.  How the body shapes the way we think - a new view on intelligence , 2006 .

[150]  J. Gold,et al.  The neural basis of decision making. , 2007, Annual review of neuroscience.

[151]  Paul F. M. J. Verschure,et al.  The Complementary Roles of Allostatic and Contextual Control Systems in Foraging Tasks , 2010, SAB.

[152]  F M J VerschurePaul Formal Minds and Biological Brains , 1993 .

[153]  Yiannis Demiris,et al.  Hierarchical attentive multiple models for execution and recognition of actions , 2006, Robotics Auton. Syst..

[154]  Tobi Delbrück,et al.  Ada: constructing a synthetic organism , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[155]  C. Koch,et al.  Multiplicative computation in a visual neuron sensitive to looming , 2002, Nature.

[156]  Olaf Sporns,et al.  Multilevel analysis of classical conditioning in a behaving real world artifact , 1995, Robotics Auton. Syst..