Boredom begets creativity: a solution to the exploitation-exploration trade-off in predictive coding

Here we investigate whether systems that minimize prediction error e.g. predictive coding, can also show creativity, or on the contrary, prediction error minimization unqualifies for the design of systems that respond in creative ways to non-recurrent problems. We argue that there is a key ingredient that has been overlooked by researchers that needs to be incorporated to understand intelligent behavior in biological and technical systems. This ingredient is boredom. We propose a mathematical model based on the Black-Scholes-Merton equation which provides mechanistic insights into the interplay between boredom and prediction pleasure as the key drivers of behavior.

[1]  Dynamic Asset Pricing Theory, Third Edition. PDF , 2016 .

[2]  W. Gerstner,et al.  Spike-Timing-Dependent Plasticity: A Comprehensive Overview , 2012, Front. Syn. Neurosci..

[3]  S. Shreve Stochastic Calculus for Finance II: Continuous-Time Models , 2010 .

[4]  G. Palm Novelty, Information and Surprise , 2012, Information Science and Statistics.

[5]  Huifang Sun,et al.  Image and Video Compression for Multimedia Engineering: Fundamentals, Algorithms, and Standards , 1999 .

[6]  Karl J. Friston,et al.  Free-Energy Minimization and the Dark-Room Problem , 2012, Front. Psychology.

[7]  Edwin T. Jaynes Prior Probabilities , 2010, Encyclopedia of Machine Learning.

[8]  Karl J. Friston,et al.  Action and behavior: a free-energy formulation , 2010, Biological Cybernetics.

[9]  Sheldon M. Ross,et al.  Stochastic Processes , 2018, Gauge Integral Structures for Stochastic Calculus and Quantum Electrodynamics.

[10]  H. Kincaid,et al.  The Oxford Handbook of Philosophy of Economics , 2009 .

[11]  T. Baldeweg Repetition effects to sounds: evidence for predictive coding in the auditory system , 2006, Trends in Cognitive Sciences.

[12]  Ricardo Sanz,et al.  On the limitations of standard statistical modeling in biological systems: a full Bayesian approach for biology. , 2013, Progress in biophysics and molecular biology.

[13]  Kiyosi Itô Stochastic Differential Equations , 2018, The Control Systems Handbook.

[14]  Raymond J. Dolan,et al.  Exploration, novelty, surprise, and free energy minimization , 2013, Front. Psychol..

[15]  J. Russell,et al.  The neurophysiological bases of emotion: An fMRI study of the affective circumplex using emotion‐denoting words , 2009, Human brain mapping.

[16]  Geoffrey E. Hinton,et al.  Keeping Neural Networks Simple , 1993 .

[17]  James Danckert,et al.  Characterizing the psychophysiological signature of boredom , 2014, Experimental Brain Research.

[18]  U. Neisser Cognitive Psychology: Classic Edition , 1967 .

[19]  Rajesh P. N. Rao,et al.  Predictive Coding , 2019, A Blueprint for the Hard Problem of Consciousness.

[20]  Mateus Joffily,et al.  Emotional Valence and the Free-Energy Principle , 2013, PLoS Comput. Biol..

[21]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[22]  D. Duffie Dynamic Asset Pricing Theory , 1992 .

[23]  Karl J. Friston,et al.  Predictive coding: an account of the mirror neuron system , 2007, Cognitive Processing.

[24]  Peter Elias,et al.  Predictive coding-I , 1955, IRE Trans. Inf. Theory.

[25]  Geoffrey E. Hinton,et al.  Keeping the neural networks simple by minimizing the description length of the weights , 1993, COLT '93.

[26]  Heather C. Lench,et al.  On the Function of Boredom , 2013, Behavioral sciences.

[27]  Karl J. Friston The history of the future of the Bayesian brain , 2012, NeuroImage.

[28]  Peter Dayan,et al.  Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems , 2001 .

[29]  Karl J. Friston,et al.  A theory of cortical responses , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[30]  D. Knill,et al.  The Bayesian brain: the role of uncertainty in neural coding and computation , 2004, Trends in Neurosciences.

[31]  Samuel M. McClure,et al.  A computational substrate for incentive salience , 2003, Trends in Neurosciences.

[32]  Peter Dayan,et al.  Dopamine: generalization and bonuses , 2002, Neural Networks.

[33]  Karl J. Friston The free-energy principle: a unified brain theory? , 2010, Nature Reviews Neuroscience.

[34]  A. Damasio,et al.  The nature of feelings: evolutionary and neurobiological origins , 2013, Nature Reviews Neuroscience.

[35]  Felipe De Brigard Predictive Memory and the Surprising Gap , 2012, Front. Psychology.

[36]  F. Black,et al.  The Pricing of Options and Corporate Liabilities , 1973, Journal of Political Economy.

[37]  Timothy D. Wilson,et al.  Just think: The challenges of the disengaged mind , 2014, Science.

[38]  A. Blaszczynski,et al.  Boredom Proneness in Pathological Gambling , 1990, Psychological reports.

[39]  A. Barto,et al.  Novelty or Surprise? , 2013, Front. Psychol..

[40]  J. Hull Options, Futures, and Other Derivatives , 1989 .

[41]  Bruno A. Olshausen,et al.  Book Review , 2003, Journal of Cognitive Neuroscience.

[42]  P. Montague,et al.  Neural Economics and the Biological Substrates of Valuation , 2002, Neuron.

[43]  Geoffrey E. Hinton,et al.  Varieties of Helmholtz Machine , 1996, Neural Networks.

[44]  Viktor Mikhaĭlovich Glushkov,et al.  An Introduction to Cybernetics , 1957, The Mathematical Gazette.

[45]  Rajesh P. N. Rao,et al.  Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. , 1999 .

[46]  Jürgen Schmidhuber,et al.  Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990–2010) , 2010, IEEE Transactions on Autonomous Mental Development.

[47]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[48]  Geoffrey E. Hinton,et al.  The Helmholtz Machine , 1995, Neural Computation.

[49]  D. Ross Integrating the Dynamics of Multiscale Economic Agency , 2009 .

[50]  J. Wagemans,et al.  Putting reward in art: A tentative prediction error account of visual art , 2011, i-Perception.

[51]  Don’t blame the economists. It is an inverse problem! , 2013 .

[52]  Anne C. Frenzel,et al.  Types of boredom: An experience sampling approach , 2014 .

[53]  M. Tribus Thermostatics and thermodynamics , 1961 .

[54]  Raymond J. Dolan,et al.  The anatomy of choice: active inference and agency , 2013, Front. Hum. Neurosci..

[55]  J. Chen The Nature of Discounting , 2009 .

[56]  S. Laughlin,et al.  Predictive coding: a fresh view of inhibition in the retina , 1982, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[57]  Geoffrey E. Hinton,et al.  A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants , 1998, Learning in Graphical Models.

[58]  P. Glimcher Understanding dopamine and reinforcement learning: The dopamine reward prediction error hypothesis , 2011, Proceedings of the National Academy of Sciences.

[59]  M. Tribus,et al.  Probability theory: the logic of science , 2003 .

[60]  G. Birkhoff Proof of the Ergodic Theorem , 1931, Proceedings of the National Academy of Sciences.

[61]  S. Shreve,et al.  Stochastic differential equations , 1955, Mathematical Proceedings of the Cambridge Philosophical Society.