Bio-inspired feedback-circuit implementation of discrete, free energy optimizing, winner-take-all computations

Bayesian inference and bounded rational decision-making require the accumulation of evidence or utility, respectively, to transform a prior belief or strategy into a posterior probability distribution over hypotheses or actions. Crucially, this process cannot be simply realized by independent integrators, since the different hypotheses and actions also compete with each other. In continuous time, this competitive integration process can be described by a special case of the replicator equation. Here we investigate simple analog electric circuits that implement the underlying differential equation under the constraint that we only permit a limited set of building blocks that we regard as biologically interpretable, such as capacitors, resistors, voltage-dependent conductances and voltage- or current-controlled current and voltage sources. The appeal of these circuits is that they intrinsically perform normalization without requiring an explicit divisive normalization. However, even in idealized simulations, we find that these circuits are very sensitive to internal noise as they accumulate error over time. We discuss in how far neural circuits could implement these operations that might provide a generic competitive principle underlying both perception and action.

[1]  Hans-Andrea Loeliger,et al.  Probability propagation and decoding in analog VLSI , 2001, IEEE Trans. Inf. Theory.

[2]  Karl J. Friston,et al.  Canonical Microcircuits for Predictive Coding , 2012, Neuron.

[3]  KongFatt Wong-Lin,et al.  Neural Circuit Dynamics Underlying Accumulation of Time-Varying Evidence During Perceptual Decision Making , 2007, Frontiers Comput. Neurosci..

[4]  Hang Zhang,et al.  Ubiquitous Log Odds: A Common Representation of Probability and Frequency Distortion in Perception, Action, and Cognition , 2012, Front. Neurosci..

[5]  Marc Harper,et al.  The Replicator Equation as an Inference Dynamic , 2009, ArXiv.

[6]  Daniel A. Braun,et al.  Thermodynamics as a theory of decision-making with information-processing costs , 2012, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[7]  D. Knill,et al.  The Bayesian brain: the role of uncertainty in neural coding and computation , 2004, Trends in Neurosciences.

[8]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[9]  Eero P. Simoncelli,et al.  Computational models of cortical visual processing. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[10]  Rajesh P. N. Rao,et al.  Bayesian brain : probabilistic approaches to neural coding , 2006 .

[11]  Matthew Cook,et al.  Real-time inference in a VLSI spiking neural network , 2012, 2012 IEEE International Symposium on Circuits and Systems.

[12]  R. Blake,et al.  Neural bases of binocular rivalry , 2006, Trends in Cognitive Sciences.

[13]  Jonathan D. Cohen,et al.  The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. , 2006, Psychological review.

[14]  Daniel Polani,et al.  Information Theory of Decisions and Actions , 2011 .

[15]  Jordi Grau-Moya,et al.  Bounded Rationality, Abstraction, and Hierarchical Decision-Making: An Information-Theoretic Optimality Principle , 2015, Front. Robot. AI.

[16]  Frances S. Chance,et al.  Gain Modulation from Background Synaptic Input , 2002, Neuron.

[17]  A. Borst Seeing smells: imaging olfactory learning in bees , 1999, Nature Neuroscience.

[18]  Daniel A. Braun,et al.  Information, Utility and Bounded Rationality , 2011, AGI.

[19]  Randolph Blake,et al.  Psychophysical magic: rendering the visible ‘invisible’ , 2005, Trends in Cognitive Sciences.

[20]  Christof Koch,et al.  Shunting Inhibition Does Not Have a Divisive Effect on Firing Rates , 1997, Neural Computation.

[21]  Karl J. Friston The free-energy principle: a rough guide to the brain? , 2009, Trends in Cognitive Sciences.

[22]  Przemyslaw Mroszczyk,et al.  The accuracy and scalability of continuous-time Bayesian inference in analogue CMOS circuits , 2014, 2014 IEEE International Symposium on Circuits and Systems (ISCAS).

[23]  Shih-Chii Liu A Winner-Take-All Circuit with Controllable Soft Max Property , 1999, NIPS.

[24]  Rajesh P. N. Rao Bayesian Computation in Recurrent Neural Circuits , 2004, Neural Computation.

[25]  Stefan Schaal,et al.  Path integral control and bounded rationality , 2011, 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL).

[26]  Daniel A. Braun,et al.  Generalized Thompson sampling for sequential decision-making and causal inference , 2013, Complex Adapt. Syst. Model..

[27]  Paul Cisek,et al.  Cortical mechanisms of action selection: the affordance competition hypothesis , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[28]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[29]  Werner Reichardt,et al.  Figure-ground discrimination by relative movement in the visual system of the fly , 2004, Biological Cybernetics.

[30]  Shawn R. Olsen,et al.  Divisive Normalization in Olfactory Population Codes , 2010, Neuron.

[31]  Xiao-Jing Wang,et al.  A Recurrent Network Mechanism of Time Integration in Perceptual Decisions , 2006, The Journal of Neuroscience.

[32]  J. Gold,et al.  Neural computations that underlie decisions about sensory stimuli , 2001, Trends in Cognitive Sciences.

[33]  M. Carandini,et al.  Summation and division by neurons in primate visual cortex. , 1994, Science.

[34]  W. Newsome,et al.  Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. , 2001, Journal of neurophysiology.

[35]  M. Carandini,et al.  Normalization as a canonical neural computation , 2011, Nature Reviews Neuroscience.

[36]  A. Yuille,et al.  Object perception as Bayesian inference. , 2004, Annual review of psychology.

[37]  Yoshua Bengio,et al.  On the Challenges of Physical Implementations of RBMs , 2013, AAAI.

[38]  Daniel A. Braun,et al.  A Minimum Relative Entropy Principle for Learning and Acting , 2008, J. Artif. Intell. Res..

[39]  J. Movshon,et al.  The analysis of visual motion: a comparison of neuronal and psychophysical performance , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[40]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[41]  Karl J. Friston,et al.  Predictive coding under the free-energy principle , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[42]  Joshua B. Tenenbaum,et al.  Multistability and Perceptual Inference , 2012, Neural Computation.

[43]  D. Vickers,et al.  Evidence for an accumulator model of psychophysical discrimination. , 1970, Ergonomics.

[44]  Wei Ji Ma,et al.  Neural coding of uncertainty and probability. , 2014, Annual review of neuroscience.

[45]  James L. McClelland,et al.  The time course of perceptual choice: the leaky, competing accumulator model. , 2001, Psychological review.

[46]  Mario Pannunzi,et al.  The Influence of Spatiotemporal Structure of Noisy Stimuli in Decision Making , 2014, PLoS Comput. Biol..

[47]  Multiplying two numbers together in your head is a difficult task if you did not learn multiplication tables as a child. On the face of it, this is somewhat surprising given the remarkable power of the brain to perform , 2010 .

[48]  Wei Ji Ma,et al.  Bayesian inference with probabilistic population codes , 2006, Nature Neuroscience.

[49]  M. Shadlen,et al.  A role for neural integrators in perceptual decision making. , 2003, Cerebral cortex.

[50]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[51]  Rajesh P. N. Rao,et al.  Implementing belief propagation in neural circuits , 2005, Neurocomputing.

[52]  Marc Toussaint,et al.  On Stochastic Optimal Control and Reinforcement Learning by Approximate Inference , 2012, Robotics: Science and Systems.

[53]  C. Shalizi Dynamics of Bayesian Updating with Dependent Data and Misspecified Models , 2009, 0901.1342.

[54]  Jordi Grau-Moya,et al.  Bounded Rational Decision-Making in Changing Environments , 2013, NIPS 2013.

[55]  Stefan J. Kiebel,et al.  Perceptual decision making: drift-diffusion model is equivalent to a Bayesian model , 2014, Front. Hum. Neurosci..

[56]  R. Briers,et al.  Ecology: From Individuals to Ecosystems , 2006 .

[57]  Timothy D. Hanks,et al.  Probabilistic Population Codes for Bayesian Decision Making , 2008, Neuron.

[58]  J. Nicholls From neuron to brain , 1976 .

[59]  Thomas L. Griffiths,et al.  Neural Implementation of Hierarchical Bayesian Inference by Importance Sampling , 2009, NIPS.

[60]  J Driver,et al.  A selective review of selective attention research from the past century. , 2001, British journal of psychology.

[61]  John L. Wyatt,et al.  The Softmax Nonlinearity: Derivation Using Statistical Mechanics and Useful Properties as a Multiterminal Analog Circuit Element , 1993, NIPS.

[62]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[63]  Marc Toussaint,et al.  Probabilistic inference for solving (PO) MDPs , 2006 .

[64]  R. Zunino,et al.  Analog implementation of the SoftMax function , 2002, 2002 IEEE International Symposium on Circuits and Systems. Proceedings (Cat. No.02CH37353).

[65]  Adele Diederich,et al.  Survey of decision field theory , 2002, Math. Soc. Sci..

[66]  Vicenç Gómez,et al.  Optimal control as a graphical model inference problem , 2009, Machine Learning.

[67]  Daniel A. Braun,et al.  Monte Carlo methods for exact & efficient solution of the generalized optimality equations , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[68]  R. Axelrod,et al.  Evolutionary Dynamics , 2004 .

[69]  D. Hammerstrom,et al.  CMOL/CMOS Implementations of Bayesian Polytree Inference: Digital and Mixed-Signal Architectures and Performance/Price , 2010, IEEE Transactions on Nanotechnology.

[70]  Xiaofeng Wang,et al.  Reinforcement Learning to Play an Optimal Nash Equilibrium in Team Markov Games , 2002, NIPS.

[71]  Jörgen W. Weibull,et al.  Evolutionary Game Theory , 1996 .

[72]  Wei Ji Ma,et al.  Spiking networks for Bayesian inference and choice , 2008, Current Opinion in Neurobiology.

[73]  Hiroo Yonezu,et al.  Analog integrated circuits for the Lotka-Volterra competitive neural networks , 1999, IEEE Trans. Neural Networks.

[74]  James L. McClelland Integrating probabilistic models of perception and interactive neural networks: a historical and tutorial review , 2013, Front. Psychol..

[75]  W. Ma Organizing probabilistic models of perception , 2012, Trends in Cognitive Sciences.

[76]  Aapo Hyvärinen,et al.  Interpreting Neural Response Variability as Monte Carlo Sampling of the Posterior , 2002, NIPS.

[77]  Emanuel Todorov,et al.  Efficient computation of optimal actions , 2009, Proceedings of the National Academy of Sciences.

[78]  J. Movshon,et al.  Linearity and Normalization in Simple Cells of the Macaque Primary Visual Cortex , 1997, The Journal of Neuroscience.

[79]  P. Holmes,et al.  The dynamics of choice among multiple alternatives , 2006 .

[80]  L. Abbott,et al.  Synaptic Depression and Cortical Gain Control , 1997, Science.

[81]  Dario L Ringach,et al.  Spontaneous and driven cortical activity: implications for computation , 2009, Current Opinion in Neurobiology.

[82]  R. Normann,et al.  The effects of background illumination on the photoresponses of red and green cones. , 1979, The Journal of physiology.

[83]  M. Begon,et al.  Ecology: From Individuals to Ecosystems , 2005 .