Neural Superstatistics: A Bayesian Method for Estimating Dynamic Models of Cognition

Mathematical models of cognition are often memoryless and ignore potential fluctuations of their parameters. However, human cognition is inherently dynamic, regardless of the reference time scale. Thus, we propose to augment mechanistic cognitive models with a temporal dimension and estimate the resulting dynamics from a superstatistics perspective. In its simplest form, such a model entails a hierarchy between a low-level observation model and a highlevel transition model. The observation model describes the local behavior of a system, and the transition model specifies how the parameters of the observation model evolve over time. To overcome the estimation challenges resulting from the complexity of superstatistical models, we develop and validate a simulation-based deep learning method for Bayesian inference, which can recover both time-varying and time-invariant parameters. We first benchmark our method against two existing frameworks capable of estimating time-varying parameters. We then apply our method to fit a dynamic version of the diffusion decision model to long time series of human response times data. Our results show that the deep learning approach is very efficient in capturing the temporal dynamics of the model. Furthermore, we show that the erroneous assumption of static or homogeneous parameters will hide important temporal information.

[1]  Stefan T. Radev,et al.  Meta-Uncertainty in Bayesian Model Comparison , 2022, ArXiv.

[2]  Stefan T. Radev,et al.  Some models are useful, but how do we know which ones? Towards a unified Bayesian model taxonomy , 2022, 2209.02439.

[3]  Stefan T. Radev,et al.  Mental speed is high until age 60 as revealed by analysis of over a million participants , 2022, Nature Human Behaviour.

[4]  Aki Vehtari,et al.  Graphical test for discrete uniformity and its applications in goodness-of-fit evaluation and multiple sample comparison , 2021, Stat. Comput..

[5]  Stefan T. Radev,et al.  BayesFlow: Learning Complex Stochastic Models With Invertible Neural Networks , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[6]  R. Kohn,et al.  Time-evolving psychological processes over repeated decisions. , 2019, Psychological review.

[7]  Aspen H. Yoo,et al.  How Working Memory and Reinforcement Learning Are Intertwined: A Cognitive, Neural, and Computational Perspective , 2021, Journal of Cognitive Neuroscience.

[8]  H. Schulze,et al.  Sleep as a random walk: a super-statistical analysis of EEG data across sleep stages , 2021, Communications biology.

[9]  Stefan T. Radev,et al.  Stability and Change in Diffusion Model Parameters over Two Years , 2021, Journal of Intelligence.

[10]  Marina Vannucci,et al.  Bayesian statistics and modelling , 2020, Nature Reviews Methods Primers.

[11]  M. Raijmakers,et al.  Hidden Markov Models of Evidence Accumulation in Speeded Decision Tasks , 2020, Computational Brain & Behavior.

[12]  Stefan T. Radev,et al.  Amortized Bayesian Model Comparison With Evidential Deep Learning , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[13]  A. Voss,et al.  Do Non-Decision Times Mediate the Association between Age and Intelligence across Different Content and Process Domains? , 2020, Journal of Intelligence.

[14]  K. Eklund,et al.  The dynamics of motivation, emotion, and task performance in simulated achievement situations , 2020 .

[15]  Luis H. Favela Cognitive science as complexity science. , 2020, Wiley interdisciplinary reviews. Cognitive science.

[16]  Gilles Louppe,et al.  The frontier of simulation-based inference , 2019, Proceedings of the National Academy of Sciences.

[17]  C. Beck,et al.  Superstatistical approach to air pollution statistics , 2019, Physical Review Research.

[18]  D. Fudenberg,et al.  Testing the drift-diffusion model , 2019, Proceedings of the National Academy of Sciences.

[19]  Anne Collins,et al.  Computational evidence for hierarchically structured reinforcement learning in humans , 2019, Proceedings of the National Academy of Sciences.

[20]  M. Betancourt,et al.  Toward a principled Bayesian workflow in cognitive science. , 2019, Psychological methods.

[21]  Yee Whye Teh,et al.  Probabilistic symmetry and invariant neural networks , 2019, J. Mach. Learn. Res..

[22]  D. Smilek,et al.  Mind wandering, motivation, and task performance over time: Evidence that motivation insulates people from the negative effects of mind wandering. , 2020 .

[23]  Ullrich Köthe,et al.  Towards end-to-end likelihood-free inference with convolutional neural networks. , 2020, The British journal of mathematical and statistical psychology.

[24]  Ullrich Köthe,et al.  Guided Image Generation with Conditional Invertible Neural Networks , 2019, ArXiv.

[25]  David S. Greenberg,et al.  Automatic Posterior Transformation for Likelihood-Free Inference , 2019, ICML.

[26]  E. Wagenmakers,et al.  Evidence Accumulation Models: Current Limitations and Future Directions , 2019, The Quantitative Methods for Psychology.

[27]  Johan Kwisthout,et al.  Cognition and intractability: a guide to classical and parameterized complexity analysis , 2019 .

[28]  Aki Vehtari,et al.  Approximate leave-future-out cross-validation for Bayesian time series models , 2019, 1902.06281.

[29]  F. Tuerlinckx,et al.  Prepaid parameter estimation without likelihoods , 2018, PLoS Comput. Biol..

[30]  Polemnia G. Amazeen,et al.  From physics to social interactions: Scientific unification via dynamics , 2018, Cognitive Systems Research.

[31]  Simon Farrell,et al.  Benchmarks for models of short-term and working memory. , 2018, Psychological bulletin.

[32]  Andrew Heathcote,et al.  Refining the Law of Practice , 2018, Psychological review.

[33]  Ben Fabry,et al.  Bayesian model selection for complex dynamic systems , 2018, Nature Communications.

[34]  Aki Vehtari,et al.  Validating Bayesian Inference Algorithms with Simulation-Based Calibration , 2018, 1804.06788.

[35]  Michael Betancourt,et al.  Calibrating Model-Based Inferences and Decisions , 2018, 1803.08393.

[36]  Michael J Frank,et al.  Within- and across-trial dynamics of human EEG reveal cooperative interplay between reinforcement learning and working memory , 2017, Proceedings of the National Academy of Sciences.

[37]  Simon Farrell,et al.  Computational Modeling of Cognition and Behavior , 2018 .

[38]  Anne E. Urai,et al.  Choice history biases subsequent evidence accumulation , 2018, bioRxiv.

[39]  Tung D Phan,et al.  The Variability Puzzle in Human Memory , 2017, bioRxiv.

[40]  T. Yarkoni,et al.  Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning , 2017, Perspectives on psychological science : a journal of the Association for Psychological Science.

[41]  Marcus Yung,et al.  Exploring the Relationship of Task Performance and Physical and Cognitive Fatigue During a Daylong Light Precision Task , 2017, Hum. Factors.

[42]  Iain Murray,et al.  Masked Autoregressive Flow for Density Estimation , 2017, NIPS.

[43]  Alex Kendall,et al.  What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? , 2017, NIPS.

[44]  A. Bunde,et al.  Superstatistical model of bacterial DNA architecture , 2017, Scientific Reports.

[45]  M. Walsh,et al.  Computational cognitive modeling of the temporal dynamics of fatigue from sleep loss , 2017, Psychonomic bulletin & review.

[46]  Jiqiang Guo,et al.  Stan: A Probabilistic Programming Language. , 2017, Journal of statistical software.

[47]  N. Daw,et al.  Reinforcement Learning and Episodic Memory in Humans and Animals: An Integrative Framework , 2017, Annual review of psychology.

[48]  Zachary C. Irving,et al.  Mind-wandering as spontaneous thought: a dynamic framework , 2016, Nature Reviews Neuroscience.

[49]  Birte U. Forstmann,et al.  A Neural Model of Mind Wandering , 2016, Trends in Cognitive Sciences.

[50]  Scott D. Brown,et al.  Diffusion Decision Model: Current Issues and History , 2016, Trends in Cognitive Sciences.

[51]  David M. Blei,et al.  Variational Inference: A Review for Statisticians , 2016, ArXiv.

[52]  H. Stanley,et al.  Universality of market superstatistics. , 2015, Physical review. E.

[53]  C. Beck,et al.  Extreme event statistics of daily rainfall: dynamical systems approach , 2015, 1508.03700.

[54]  C. Beck,et al.  Superstatistical analysis of sea-level fluctuations , 2014, 1405.6007.

[55]  Alexander Robitzsch,et al.  Dynamical systems analysis applied to working memory data , 2014, Front. Psychol..

[56]  Michael X Cohen,et al.  Analyzing Neural Time Series Data: Theory and Practice , 2014 .

[57]  Michael J. Richardson,et al.  Modeling the Dynamics of Risky Choice , 2013 .

[58]  A. Voss,et al.  Diffusion Models in Experimental Psychology , 2013 .

[59]  J. Holden,et al.  Dynamics of cognition. , 2012, Wiley interdisciplinary reviews. Cognitive science.

[60]  Bernhard Schölkopf,et al.  A Kernel Two-Sample Test , 2012, J. Mach. Learn. Res..

[61]  R. Ratcliff,et al.  Diffusion model for one-choice reaction-time tasks and the cognitive effects of sleep deprivation , 2011, Proceedings of the National Academy of Sciences.

[62]  Andrew Gelman,et al.  Handbook of Markov Chain Monte Carlo , 2011 .

[63]  Stefan Thurner,et al.  Generalized entropies and the transformation group of superstatistics , 2011, Proceedings of the National Academy of Sciences.

[64]  Beatrix Vereijken,et al.  Interaction-dominant dynamics in human cognition: beyond 1/f(alpha) fluctuation. , 2010, Journal of experimental psychology. General.

[65]  Christian Beck,et al.  Superstatistical fluctuations in time series: applications to share-price dynamics and turbulence. , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[66]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[67]  Andreas Voss,et al.  Fast-dm: A free program for efficient diffusion model analysis , 2007, Behavior research methods.

[68]  Kristina M. Visscher,et al.  The neural bases of momentary lapses in attention , 2006, Nature Neuroscience.

[69]  A. Diederich,et al.  Modeling the effects of payoff on response bias in a perceptual discrimination task: Bound-change, drift-rate-change, or two-stage-processing hypothesis , 2006, Perception & psychophysics.

[70]  R. Ratcliff,et al.  Estimation and interpretation of 1/fα noise in human cognition , 2004 .

[71]  G. V. van Orden,et al.  Self-organization of cognitive performance. , 2003, Journal of experimental psychology. General.

[72]  R. Ratcliff,et al.  Estimating parameters of the diffusion model: Approaches to dealing with contaminant reaction times and parameter variability , 2002, Psychonomic bulletin & review.

[73]  D. Gilden Cognitive emissions of 1/f noise. , 2001, Psychological review.

[74]  Jürgen Schmidhuber,et al.  Learning to Forget: Continual Prediction with LSTM , 2000, Neural Computation.

[75]  Jeffrey N. Rouder,et al.  Modeling Response Times for Two-Choice Decisions , 1998 .

[76]  S. Eddy Hidden Markov models. , 1996, Current opinion in structural biology.

[77]  P. Phillips,et al.  Vector autoregression and causality: a theoretical overview and simulation study , 1994 .

[78]  Roger Ratcliff,et al.  A Theory of Memory Retrieval. , 1978 .