Complex Probabilistic Inference

Our understanding of probabilistic inference in the brain has progressed rapidly. However, there remains a big gap between the relatively simple probabilistic inference problems facing low-level sensory systems and the intractably complex problems facing high-level cognitive systems. Psychologists have begun exploring cognitively plausible algorithms for approximately solving complex inference problems. We review recent attempts to connect these algorithmic accounts to neural circuit mechanisms, and argue that neural mechanisms for solving low-level sensory inference problems can be extended to tackle complex inference problems.

[1]  Charles Kemp,et al.  How to Grow a Mind: Statistics, Structure, and Abstraction , 2011, Science.

[2]  Chong Wang,et al.  Stochastic variational inference , 2012, J. Mach. Learn. Res..

[3]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[4]  Ilker Yildirim.  Efficient analysis-by-synthesis in vision : A computational framework , behavioral tests , and comparison with neural representations , 2015 .

[5]  Wolfgang Maass,et al.  Probabilistic Inference in General Graphical Models through Sampling in Stochastic Networks of Spiking Neurons , 2011, PLoS Comput. Biol..

[6]  Christian K. Machens,et al.  Predictive Coding of Dynamical Variables in Balanced Spiking Networks , 2013, PLoS Comput. Biol..

[7]  Rajesh P. N. Rao.  Bayesian Computation in Recurrent Neural Circuits , 2004, Neural Computation.

[8]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[9]  Karl J. Friston,et al.  Canonical Microcircuits for Predictive Coding , 2012, Neuron.

[10]  W. Levelt.  On binocular rivalry , 1965 .

[11]  Mark Steyvers,et al.  Topics in semantic representation. , 2007, Psychological review.

[12]  Dileep George,et al.  Towards a Mathematical Theory of Cortical Micro-circuits , 2009, PLoS Comput. Biol..

[13]  Wei Ji Ma,et al.  Bayesian inference with probabilistic population codes , 2006, Nature Neuroscience.

[14]  G. DeAngelis,et al.  Neural correlates of multisensory cue integration in macaque MSTd , 2008, Nature Neuroscience.

[15]  R. Blake.  © 2001 Kluwer Academic Publishers. Printed in the Netherlands. 5 A Primer on Binocular Rivalry, Including Current Controversies , 2000 .

[16]  Neil A. Macmillan,et al.  Detection Theory: A User's Guide , 1991 .

[17]  Robert A. Legenstein,et al.  Ensembles of Spiking Neurons with Noise Support Optimal Probabilistic Inference in a Dynamically Changing Environment , 2014, PLoS Comput. Biol..

[18]  Shimon Ullman,et al.  Cortical Circuitry Implementing Graphical Models , 2009, Neural Computation.

[19]  Rajesh P. N. Rao,et al.  Neurons as Monte Carlo Samplers: Bayesian Inference and Learning in Spiking Networks , 2014, NIPS.

[20]  József Fiser,et al.  Perceptual Decision-Making as Probabilistic Inference by Neural Sampling , 2014, Neuron.

[21]  Alexandre Pouget,et al.  Complex Inference in Neural Circuits with Probabilistic Population Codes and Topic Models , 2012, NIPS.

[22]  Thomas L. Griffiths,et al.  Modeling the effects of memory on human online sentence processing with particle filters , 2008, NIPS.

[23]  Christopher D. Manning,et al.  Probabilistic models of language processing and acquisition , 2006, Trends in Cognitive Sciences.

[24]  Tom Minka,et al.  Expectation Propagation for approximate Bayesian inference , 2001, UAI.

[25]  Peter Dayan,et al.  Optimal Recall from Bounded Metaplastic Synapses: Predicting Functional Adaptations in Hippocampal Area CA3 , 2014, PLoS Comput. Biol..

[26]  Cristina Savin,et al.  Spatio-temporal Representations of Uncertainty in Spiking Neural Networks , 2014, NIPS.

[27]  Geoffrey E. Hinton,et al.  The Helmholtz Machine , 1995, Neural Computation.

[28]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[29]  N. Logothetis,et al.  Multistable phenomena: changing views in perception , 1999, Trends in Cognitive Sciences.

[30]  S. Denison,et al.  Rational variability in children’s causal inferences: The Sampling Hypothesis , 2013, Cognition.

[31]  R. Shepard,et al.  Toward a universal law of generalization for psychological science. , 1987, Science.

[32]  J. Gold,et al.  Banburismus and the Brain Decoding the Relationship between Sensory Stimuli, Decisions, and Reward , 2002, Neuron.

[33]  K. Holyoak,et al.  Causal learning and inference as a rational process: the new synthesis. , 2011, Annual review of psychology.

[34]  Charles H. Anderson,et al.  BASIC ELEMENTS OF BIOLOGICAL COMPUTATIONAL SYSTEMS , 1994 .

[35]  Michael N. Shadlen,et al.  Probabilistic reasoning by neurons , 2007, Nature.

[36]  Scott D. Brown,et al.  Detecting and predicting changes , 2009, Cognitive Psychology.

[37]  Michael C. Frank,et al.  Modeling human performance in statistical word segmentation , 2010, Cognition.

[38]  Thomas L. Griffiths,et al.  A Rational Analysis of Rule-Based Concept Learning , 2008, Cogn. Sci..

[39]  A. Pouget,et al.  Probabilistic brains: knowns and unknowns , 2013, Nature Neuroscience.

[40]  Richard F. Murray,et al.  Posterior Probability Matching and Human Perceptual Decision Making , 2015, PLoS Comput. Biol..

[41]  Anthony J. Movshon,et al.  Optimal representation of sensory information by neural populations , 2006, Nature Neuroscience.

[42]  Ulrik R. Beierholm,et al.  Probability Matching as a Computational Strategy Used in Perception , 2010, PLoS Comput. Biol..

[43]  P Wenderoth,et al.  Determinants of Fusion of Dichoptically Presented Orthogonal Gratings , 1999, Perception.

[44]  D. Knill,et al.  Bayesian sampling in visual perception , 2011, Proceedings of the National Academy of Sciences.

[45]  Noah D. Goodman,et al.  Amortized Inference in Probabilistic Reasoning , 2014, CogSci.

[46]  A. Yuille,et al.  Object perception as Bayesian inference. , 2004, Annual review of psychology.

[47]  A. Doucet,et al.  Particle Markov chain Monte Carlo methods , 2010 .

[48]  Nir Vulkan.  An Economist's Perspective on Probability Matching , 2000 .

[49]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[50]  Shakir Mohamed,et al.  Variational Inference with Normalizing Flows , 2015, ICML.

[51]  Daan Wierstra,et al.  Stochastic Backpropagation and Approximate Inference in Deep Generative Models , 2014, ICML.

[52]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine-mediated learning.

[53]  W. Richards,et al.  Perception as Bayesian Inference , 2008 .

[54]  Adam N Sanborn,et al.  Rational approximations to rational models: alternative algorithms for category learning. , 2010, Psychological review.

[55]  H. Wilson,et al.  Dynamics of travelling waves in visual perception , 2001, Nature.

[56]  A. Pouget,et al.  Marginalization in Neural Circuits with Divisive Normalization , 2011, The Journal of Neuroscience.

[57]  Wolfram Burgard,et al.  Robust Monte Carlo localization for mobile robots , 2001, Artif. Intell..

[58]  S. Chib,et al.  Understanding the Metropolis-Hastings Algorithm , 1995 .

[59]  Alan A. Stocker,et al.  Characterizing the Impact of Category Uncertainty on Human Auditory Categorization Behavior , 2014, PLoS Comput. Biol..

[60]  Thomas L. Griffiths,et al.  One and Done? Optimal Decisions From Very Few Samples , 2014, Cogn. Sci..

[61]  Ryota Kanai,et al.  Distance in feature space determines exclusivity in visual rivalry , 2007, Vision Research.

[62]  A. Pouget,et al.  Not Noisy, Just Wrong: The Role of Suboptimal Inference in Behavioral Variability , 2012, Neuron.

[63]  Max Welling,et al.  Markov Chain Monte Carlo and Variational Inference: Bridging the Gap , 2014, ICML.

[64]  George A. Alvarez,et al.  Explaining human multiple object tracking as resource-constrained approximate inference in a dynamic probabilistic model , 2009, NIPS.

[65]  Adam N. Sanborn.  Types of approximation for probabilistic cognition: Sampling and variational , 2017, Brain and Cognition.

[66]  Samuel J. Gershman,et al.  Computational rationality: A converging paradigm for intelligence in brains, minds, and machines , 2015, Science.

[67]  J. Tenenbaum,et al.  Generalization, similarity, and Bayesian inference. , 2001, The Behavioral and brain sciences.

[68]  Luigi Acerbi,et al.  On the Origins of Suboptimality in Human Probabilistic Inference , 2014, PLoS Comput. Biol..

[69]  Joshua B. Tenenbaum,et al.  Multistability and Perceptual Inference , 2012, Neural Computation.

[70]  Wolfgang Maass,et al.  Neural Dynamics as Sampling: A Model for Stochastic Computation in Recurrent Networks of Spiking Neurons , 2011, PLoS Comput. Biol..

[71]  Guillaume Hennequin,et al.  Fast Sampling-Based Inference in Balanced Neuronal Networks , 2014, NIPS.

[72]  Pascal Mamassian,et al.  Observer biases in the 3D interpretation of line drawings , 1998, Vision Research.

[73]  Noah D. Goodman,et al.  Amortized Hypothesis Generation , 2017, bioRxiv.