Bounded Rational Decision-Making in Feedforward Neural Networks

Bounded rational decision-makers transform sensory input into motor output under limited computational resources. Mathematically, such decision-makers can be modeled as information-theoretic channels with limited transmission rate. Here, we apply this formalism for the first time to multilayer feedforward neural networks. We derive synaptic weight update rules for two scenarios, where either each neuron is considered as a bounded rational decision-maker or the network as a whole. In the update rules, bounded rationality translates into information-theoretically motivated types of regularization in weight space. In experiments on the MNIST benchmark classification task for handwritten digits, we show that such information-theoretic regularization successfully prevents overfitting across different architectures and attains results that are competitive with other recent techniques like dropout, dropconnect and Bayes by backprop, for both ordinary and convolutional neural networks.

[1]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[2]  Imre Csiszár,et al.  On the computation of rate-distortion functions (Corresp.) , 1974, IEEE Trans. Inf. Theory.

[3]  Yann LeCun,et al.  Regularization of Neural Networks using DropConnect , 2013, ICML.

[4]  Daniel Polani,et al.  Information Theory of Decisions and Actions , 2011 .

[5]  Patrice Y. Simard,et al.  Best practices for convolutional neural networks applied to visual document analysis , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[6]  Suguru Arimoto,et al.  An algorithm for computing the capacity of arbitrary discrete memoryless channels , 1972, IEEE Trans. Inf. Theory.

[7]  B. Jones.  BOUNDED RATIONALITY , 1999 .

[8]  Naftali Tishby,et al.  The information bottleneck method , 2000, ArXiv.

[9]  Julien Cornebise,et al.  Weight Uncertainty in Neural Networks , 2015, ArXiv.

[10]  Daniel A. Braun,et al.  Thermodynamics as a theory of decision-making with information-processing costs , 2012, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[11]  Karl J. Friston.  The free-energy principle: a unified brain theory? , 2010, Nature Reviews Neuroscience.

[12]  Emanuel Todorov,et al.  Efficient computation of optimal actions , 2009, Proceedings of the National Academy of Sciences.

[13]  Christopher A. Sims,et al.  RATIONAL INATTENTION AND MONETARY ECONOMICS , 2010 .

[14]  Susanne Still,et al.  Information-theoretic approach to interactive learning , 2007, 0709.1948.

[15]  David H. Wolpert,et al.  Information Theory - The Bridge Connecting Bounded Rational Game Theory and Statistical Physics , 2004, ArXiv.

[16]  Kee-Eung Kim,et al.  Information-Theoretic Bounded Rationality , 2015, ArXiv.

[17]  Yasemin Altun,et al.  Relative Entropy Policy Search , 2010 .

[18]  Richard E. Blahut,et al.  Computation of channel capacity and rate-distortion functions , 1972, IEEE Trans. Inf. Theory.

[19]  Daniel A. Braun,et al.  A Reward-Maximizing Spiking Neuron as a Bounded Rational Decision Maker , 2015, Neural Computation.

[20]  Lars-Göran Mattsson,et al.  Probabilistic choice and procedurally bounded rationality , 2002, Games Econ. Behav..

[21]  Naftali Tishby,et al.  Trading Value and Information in MDPs , 2012 .

[22]  Vicenç Gómez,et al.  Optimal control as a graphical model inference problem , 2009, Machine Learning.

[23]  Jordi Grau-Moya,et al.  Bounded Rationality, Abstraction, and Hierarchical Decision-Making: An Information-Theoretic Optimality Principle , 2015, Front. Robot. AI.

[24]  Aaron D. Wyner,et al.  Coding Theorems for a Discrete Source With a Fidelity CriterionInstitute of Radio Engineers, International Convention Record, vol. 7, 1959. , 1993 .

[25]  Susanne Still,et al.  LOSSY IS LAZY , 2015 .

[26]  E. Rowland.  Theory of Games and Economic Behavior , 1946, Nature.

[27]  Marc Toussaint,et al.  On Stochastic Optimal Control and Reinforcement Learning by Approximate Inference , 2012, Robotics: Science and Systems.

[28]  Klaus-Robert Müller,et al.  Efficient BackProp , 2012, Neural Networks: Tricks of the Trade.

[29]  Samuel J. Gershman,et al.  Computational rationality: A converging paradigm for intelligence in brains, minds, and machines , 2015, Science.

[30]  José Carlos Príncipe,et al.  Rate-Distortion Auto-Encoders , 2013, ICLR.

[31]  Clément Farabet,et al.  Torch7: A Matlab-like Environment for Machine Learning , 2011, NIPS 2011.

[32]  Xiaohui Xie,et al.  Learning in neural networks by reinforcement of irregular spiking. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[33]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..