A Programmer-Interpreter Neural Network Architecture for Prefrontal Cognitive Control

There is wide consensus that the prefrontal cortex (PFC) is able to exert cognitive control on behavior by biasing processing toward task-relevant information and by modulating response selection. This idea is typically framed in terms of top-down influences within a cortical control hierarchy, where prefrontal-basal ganglia loops gate multiple input-output channels, which in turn can activate or sequence motor primitives expressed in (pre-)motor cortices. Here we advance a new hypothesis, based on the notion of programmability and an interpreter-programmer computational scheme, on how the PFC can flexibly bias the selection of sensorimotor patterns depending on internal goal and task contexts. In this approach, multiple elementary behaviors representing motor primitives are expressed by a single multi-purpose neural network, which is seen as a reusable area of "recycled" neurons (interpreter). The PFC thus acts as a "programmer" that, without modifying the network connectivity, feeds the interpreter networks with specific input parameters encoding the programs (corresponding to network structures) to be interpreted by the (pre-)motor areas. Our architecture is validated in a standard test for executive function: the 1-2-AX task. Our results show that this computational framework provides a robust, scalable and flexible scheme that can be iterated at different hierarchical layers, supporting the realization of multiple goals. We discuss the plausibility of the "programmer-interpreter" scheme to explain the functioning of prefrontal-(pre)motor cortical hierarchies.

[1]  Xiao-Jing Wang,et al.  The importance of mixed selectivity in complex cognitive tasks , 2013, Nature.

[2]  Hava T. Siegelmann,et al.  Neural networks and analog computation - beyond the Turing limit , 1999, Progress in theoretical computer science.

[3]  C. Summerfield,et al.  An information theoretical approach to prefrontal executive function , 2007, Trends in Cognitive Sciences.

[4]  S. Hurley The shared circuits model (SCM): how control, mirroring, and simulation can enable imitation, deliberation, and mindreading. , 2008, The Behavioral and brain sciences.

[5]  Giovanni Pezzulo,et al.  An Active Inference view of cognitive control , 2012, Front. Psychology.

[6]  Silvia Tolu,et al.  Adaptive cerebellar Spiking Model Embedded in the Control Loop: Context Switching and Robustness against noise , 2011, Int. J. Neural Syst..

[7]  Giovanni Pezzulo,et al.  Divide et impera: subgoaling reduces the complexity of probabilistic inference and problem solving , 2015, Journal of The Royal Society Interface.

[8]  Max A. Viergever,et al.  Dynamics of Collective Multi-stability in Models of Multi-Unit neuronal Systems , 2014, Int. J. Neural Syst..

[9]  Roberto Prevete,et al.  How and over what timescales does neural reuse actually occur? , 2010, Behavioral and Brain Sciences.

[10]  A. Turing On Computable Numbers, with an Application to the Entscheidungsproblem. , 1937 .

[11]  Jun Tani,et al.  Emergence of Functional Hierarchy in a Multiple Timescale Neural Network Model: A Humanoid Robot Experiment , 2008, PLoS Comput. Biol..

[12]  Stiliyan Kalitzin,et al.  Multiple oscillatory States in Models of Collective neuronal Dynamics , 2014, Int. J. Neural Syst..

[13]  Michael I. Jordan Attractor dynamics and parallelism in a connectionist sequential machine , 1990 .

[14]  W. Senn,et al.  Multiple time scales of temporal response in pyramidal and fast spiking cortical neurons. , 2006, Journal of neurophysiology.

[15]  Chris Eliasmith,et al.  Neural Engineering: Computation, Representation, and Dynamics in Neurobiological Systems , 2004, IEEE Transactions on Neural Networks.

[16]  Chris Eliasmith,et al.  A Unified Approach to Building and Controlling Spiking Attractor Networks , 2005, Neural Computation.

[17]  Paul Miller,et al.  Inhibitory control by an integral feedback signal in prefrontal cortex: a model of discrimination between sequential stimuli. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[18]  J. Hopfield,et al.  Computing with neural circuits: a model. , 1986, Science.

[19]  Randall D. Beer,et al.  On the Dynamics of Small Continuous-Time Recurrent Neural Networks , 1995, Adapt. Behav..

[20]  David Sussillo,et al.  Neural circuits as computational dynamical systems , 2014, Current Opinion in Neurobiology.

[21]  J. Fuster The Prefrontal Cortex , 1997 .

[22]  Matthew M Botvinick,et al.  Short-term memory for serial order: a recurrent neural network model. , 2006, Psychological review.

[23]  Jonathan D. Cohen,et al.  Prefrontal cortex and flexible cognitive control: rules without symbols. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[24]  Jian-Xin Xu,et al.  Biological modeling of complex chemotaxis behaviors for C. elegans under speed regulation—a dynamic neural networks approach , 2013, Journal of Computational Neuroscience.

[25]  Berj L. Bardakjian,et al.  Responsive Neuromodulators Based on Artificial Neural Networks Used to Control Seizure-like Events in a Computational Model of Epilepsy , 2011, Int. J. Neural Syst..

[26]  Michael J. Frank,et al.  Interactions between frontal cortex and basal ganglia in working memory: A computational model , 2001, Cognitive, affective & behavioral neuroscience.

[27]  Giovanni Pezzulo,et al.  Mental imagery in the navigation domain: a computational model of sensory-motor simulation mechanisms , 2013, Adapt. Behav..

[28]  S Dehaene,et al.  A neuronal model of a global workspace in effortful cognitive tasks. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[29]  J. Duncan The multiple-demand (MD) system of the primate brain: mental programs for intelligent behaviour , 2010, Trends in Cognitive Sciences.

[30]  Masakazu Konishi,et al.  Robustness of Multiplicative Processes in Auditory Spatial Tuning , 2004, The Journal of Neuroscience.

[31]  Asim Roy,et al.  Connectionism, Controllers, and a Brain Theory , 2008, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[32]  Karl J. Friston,et al.  Structural and Functional Brain Networks: From Connections to Cognition , 2013, Science.

[33]  Giuseppe Trautteur,et al.  Computational virtuality in biological systems , 2009, Theor. Comput. Sci..

[34]  M. Botvinick,et al.  Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective , 2009, Cognition.

[35]  Jürgen Schmidhuber,et al.  Learning to Control Fast-Weight Memories: An Alternative to Dynamic Recurrent Networks , 1992, Neural Computation.

[36]  Giancarlo Ferrigno,et al.  Error mapping controller: a closed loop neuroprosthesis controlled by artificial neural networks , 2006, Journal of NeuroEngineering and Rehabilitation.

[37]  Antoni Morro,et al.  Studying the Role of Synchronized and Chaotic Spiking Neural Ensembles in Neural Information Processing , 2014, Int. J. Neural Syst..

[38]  C. Gilbert,et al.  Brain States: Top-Down Influences in Sensory Processing , 2007, Neuron.

[39]  R. Passingham,et al.  The Neurobiology of the Prefrontal Cortex: Anatomy, Evolution, and the Origin of Insight , 2012 .

[40]  Roberto Prevete,et al.  Programming in the brain: a neural network theoretical framework , 2012, Connect. Sci..

[41]  Michael L. Anderson Neural reuse: A fundamental organizational principle of the brain , 2010, Behavioral and Brain Sciences.

[42]  David S. Touretzky,et al.  BoltzCONS: Dynamic Symbol Structures in a Connectionist Network , 1990, Artif. Intell..

[43]  M. Botvinick Hierarchical models of behavior and prefrontal function , 2008, Trends in Cognitive Sciences.

[44]  Michael A. Arbib,et al.  Schema design and implementation of the grasp-related mirror neuron system , 2002, Biological Cybernetics.

[45]  E. Marder Neuromodulation of Neuronal Circuits: Back to the Future , 2012, Neuron.

[46]  Jun Tani,et al.  Generalization in Learning Multiple Temporal Patterns Using RNNPB , 2004, ICONIP.

[47]  E. Miller,et al.  An integrative theory of prefrontal cortex function. , 2001, Annual review of neuroscience.

[48]  J. Fodor The Modularity of mind. An essay on faculty psychology , 1986 .

[49]  David Badre,et al.  Cognitive control, hierarchy, and the rostro–caudal organization of the frontal lobes , 2008, Trends in Cognitive Sciences.

[50]  Jun Tani,et al.  Motor primitive and sequence self-organization in a hierarchical recurrent neural network , 2004, Neural Networks.

[51]  M. Frank,et al.  Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 1: computational analysis. , 2012, Cerebral cortex.

[52]  J. Fodor,et al.  The Modularity of Mind: An Essay on Faculty Psychology , 1984 .

[53]  Michael J. Frank,et al.  Making Working Memory Work: A Computational Model of Learning in the Prefrontal Cortex and Basal Ganglia , 2006, Neural Computation.

[54]  A. Treves,et al.  Hippocampal remapping and grid realignment in entorhinal cortex , 2007, Nature.

[55]  Roberto Prevete,et al.  A Robotic Scenario for Programmable Fixed-Weight Neural Networks Exhibiting Multiple Behaviors , 2011, ICANNGA.

[56]  C. Lee Giles,et al.  Learning and Extracting Finite State Automata with Second-Order Recurrent Neural Networks , 1992, Neural Computation.

[57]  R. French,et al.  Catastrophic Forgetting in Connectionist Networks: Causes, Consequences and Solutions , 1994 .

[58]  Walter Senn,et al.  Code-Specific Learning Rules Improve Action Selection by Populations of Spiking Neurons , 2014, Int. J. Neural Syst..

[59]  Aude Billard,et al.  Parallel and distributed neural models of the ideomotor principle: An investigation of imitative cortical pathways , 2006, Neural Networks.

[60]  Michael I. Jordan,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1994, Neural Computation.

[61]  Sander M. Bohte,et al.  Computing with Spiking Neuron Networks , 2012, Handbook of Natural Computing.

[62]  Marco Zorzi,et al.  The Role of dopamine in the Maintenance of Working Memory in prefrontal Cortex Neurons: Input-Driven versus Internally-Driven Networks , 2010, Int. J. Neural Syst..

[63]  Matthijs A. A. van der Meer,et al.  Internally generated sequences in learning and executing goal-directed behavior , 2014, Trends in Cognitive Sciences.

[64]  R. Andersen,et al.  Multimodal representation of space in the posterior parietal cortex and its use in planning movements. , 1997, Annual review of neuroscience.

[65]  Panos E. Trahanias,et al.  Self-organizing high-order cognitive functions in artificial agents: Implications for possible prefrontal cortex mechanisms , 2012, Neural Networks.

[66]  Garrison W. Cottrell,et al.  Towards Instructable Connectionist Systems , 1995 .

[67]  Karl J. Friston,et al.  A Hierarchy of Time-Scales and the Brain , 2008, PLoS Comput. Biol..

[68]  Danil V. Prokhorov,et al.  Adaptive behavior with fixed weights in RNN: an overview , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[69]  G. Pezzulo,et al.  Human Sensorimotor Communication: A Theory of Signaling in Online Social Interactions , 2013, PloS one.

[70]  G. Rizzolatti,et al.  Functional organization of inferior area 6 in the macaque monkey , 2004, Experimental Brain Research.

[71]  Viktor K. Jirsa,et al.  Time Scale Hierarchies in the Functional Organization of Complex Behaviors , 2011, PLoS Comput. Biol..

[72]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[73]  L. Abbott,et al.  A model of multiplicative neural responses in parietal cortex. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[74]  R. French Catastrophic forgetting in connectionist networks , 1999, Trends in Cognitive Sciences.

[75]  Wolfgang M. Pauli,et al.  Computational models of cognitive control , 2010, Current Opinion in Neurobiology.

[76]  John S. Conery,et al.  A Neural Network Model of Chemotaxis Predicts Functions of Synaptic Connections in the Nematode Caenorhabditis elegans , 2004, Journal of Computational Neuroscience.

[77]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[78]  Cori Bargmann Beyond the connectome: How neuromodulators shape neural circuits , 2012, BioEssays : news and reviews in molecular, cellular and developmental biology.

[79]  Mauro Ursino,et al.  A Multi-Layer Neural-Mass Model for Learning Sequences using Theta/Gamma oscillations , 2013, Int. J. Neural Syst..