Prediction and Control in a Dynamic Environment

The present study compared the accuracy of cue-outcome knowledge gained during prediction-based and control-based learning in stable and unstable dynamic environments. Participants either learnt to make cue-interventions in order to control an outcome, or learnt to predict the outcome from observing changes to the cue values. Study 1 (N = 60) revealed that in tests of control, after a short period of familiarization, performance of Predictors was equivalent to Controllers. Study 2 (N = 28) showed that Controllers showed equivalent task knowledge when to compared to Predictors. Though both Controllers and Predictors showed good performance at test, overall Controllers showed an advantage. The cue-outcome knowledge acquired during learning was sufficiently flexible to enable successful transfer to tests of control and prediction.

[1]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[2]  Z. Dienes,et al.  The role of specific instances in controlling a dynamic system , 1995 .

[3]  Michael E. Doherty,et al.  Feedback effects in a metric multiple-cue probability learning task. , 1976 .

[4]  M. Gluck,et al.  Probabilistic classification learning in amnesia. , 1994, Learning & memory.

[5]  D. Broadbent,et al.  Interactive tasks and the implicit‐explicit distinction , 1988 .

[6]  Cleotilde Gonzalez,et al.  Instance-based learning in dynamic decision making , 2003 .

[7]  J. Funke.  Microworlds based on linear equation systems: A new approach to complex problem solving and experimental results , 1993 .

[8]  Jennifer A. Mangels,et al.  A Neostriatal Habit Learning System in Humans , 1996, Science.

[9]  Joshua B. Tenenbaum,et al.  Inferring causal networks from observations and interventions , 2003, Cogn. Sci..

[10]  D. Shanks,et al.  Learning strategies in amnesia , 2008, Neuroscience & Biobehavioral Reviews.

[11]  P. Dayan,et al.  Decision theory, reinforcement learning, and the brain , 2008, Cognitive, affective & behavioral neuroscience.

[12]  M. Jahanshahi,et al.  Patients with Parkinson's disease learn to control complex systems via procedural as well as non-procedural learning , 2008, Neuropsychologia.

[13]  N. Castellan,et al.  An hypothesis generation model for judgment in nonmetric multiple-cue probability learning , 1973 .

[14]  Magda Osman,et al.  Observation Can Be as Effective as Action in Problem Solving , 2008, Cogn. Sci..

[15]  Marvin S. Cohen,et al.  Metarecognition in Time-Stressed Decision Making: Recognizing, Critiquing, and Correcting , 1996, Hum. Factors.

[16]  B. Brehmer.  Dynamic decision making: human control of complex systems. , 1992, Acta psychologica.

[17]  D. Shanks,et al.  Learning in a changing environment. , 2010, Journal of experimental psychology. General.

[18]  J. R R. Leigh.  Control Theory: A Guided Tour , 1992 .

[19]  D. Lagnado,et al.  Spontaneous Causal Learning While Controlling A Dynamic System , 2010 .

[20]  E. Salas,et al.  Taking stock of naturalistic decision making , 2001 .

[21]  Tom Burr,et al.  Causation, Prediction, and Search , 2003, Technometrics.

[22]  Y. S. Lee,et al.  Effects of learning contexts on implicit and explicit learning , 1995, Memory & cognition.

[23]  P. Dayan,et al.  States versus Rewards: Dissociable Neural Prediction Error Signals Underlying Model-Based and Model-Free Reinforcement Learning , 2010, Neuron.

[24]  Gary Klein,et al.  Developing Expertise in Decision Making , 1997 .

[25]  S. Sloman,et al.  The advantage of timely intervention. , 2004, Journal of experimental psychology. Learning, memory, and cognition.

[26]  Kenneth R. Hammond,et al.  Negative effects of outcome-feedback in multiple-cue probability learning. , 1973 .

[27]  Jared Freeman,et al.  Metarecognition in Time-Stressed Decision Making: Recognizing, Critiquing, and Correcting , 1996, Hum. Factors.

[28]  Charles A.J. Vlek,et al.  Aspects of suboptimality in a multidimensional probabilistic information processing task , 1970 .

[29]  H. Rothstein.  The effects of time pressure on judgment in multiple cue probability learning , 1986 .

[30]  Penelope M. Sanderson,et al.  Verbalizable knowledge and skilled task performance: Association, dissociation, and mental models. , 1989 .

[31]  Konrad Paul Kording,et al.  Review TRENDS in Cognitive Sciences Vol.10 No.7 July 2006 Special Issue: Probabilistic models of cognition Bayesian decision theory in sensorimotor control , 2022 .

[32]  B. Burns,et al.  The Quarterly Journal of Experimental Psychology Section A: Human Experimental Psychology Goal Specificity Effects on Hypothesis Testing in Problem Solving , 2022 .

[33]  D. Broadbent,et al.  On the Relationship between Task Performance and Associated Verbalizable Knowledge , 1984 .

[34]  Faison P. Gibson,et al.  Learning and transfer in dynamic decision environments , 2007, Comput. Math. Organ. Theory.

[35]  K. Holyoak,et al.  The Impact of Goal Specificity on Strategy Use and the Acquisition of Problem Structure , 1996 .

[36]  M. Osman.  Controlling uncertainty: a review of human behavior in complex dynamic environments. , 2010, Psychological bulletin.

[37]  M. Steyvers,et al.  The dynamics of experimentally induced criterion shifts. , 2005, Journal of experimental psychology. Learning, memory, and cognition.

[38]  M. Osman.  Positive transfer and negative transfer/antilearning of problem-solving skills. , 2008, Journal of experimental psychology. General.

[39]  Vancouver,et al.  Analyzing Goal-Striving Processes and a Test of the Generalizability of Perceptual Control Theory. , 2000, Organizational behavior and human decision processes.

[40]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[41]  P. Juslin,et al.  On the role of causal intervention in multiple-cue judgment: positive and negative effects on learning. , 2006, Journal of experimental psychology. Learning, memory, and cognition.

[42]  Amanda Price.  Distinguishing the contributions of implicit and explicit processes to performance of the weather prediction task , 2009, Memory & cognition.

[43]  M. Gluck,et al.  Interactive memory systems in the human brain , 2001, Nature.

[44]  N. Castellan.  Multiple-cue probability learning with irrelevant cues☆ , 1973 .

[45]  Cleotilde Gonzalez,et al.  Instance-based learning in dynamic decision making , 2003, Cogn. Sci..

[46]  D. Plaut,et al.  Learning in Dynamic Decision Tasks: Computational Model and Empirical Evidence , 1997 .

[47]  P. Frensch,et al.  Complex problem solving : the European perspective , 1995 .

[48]  Z Dienes,et al.  The Role of Implicit Memory in Controlling a Dynamic System , 1998, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[49]  Ron Sun,et al.  From implicit skills to explicit knowledge: a bottom-up model of skill learning , 2001, Cogn. Sci..

[50]  Jayashankar M. Swaminathan,et al.  Is timely information always better? The effect of feedback frequency on decision making ☆ , 2009 .

[51]  D. Broadbent.  Levels, Hierarchies, and the Locus of Control* , 1977 .

[52]  P. Muchinsky,et al.  Human inference behavior in abstract and meaningful environments , 1975 .

[53]  Natassia Goode,et al.  You need to know: There is a causal relationship between structural knowledge and control performance in complex problem solving tasks , 2010 .

[54]  Magda Osman,et al.  Seeing is as Good as Doing , 2008, J. Probl. Solving.

[55]  Joachim Funke,et al.  Dealing with dynamic systems: Research strategy, diagnostic approach and experimental results , 1992 .

[56]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[57]  J. Meditch,et al.  Applied optimal control , 1972, IEEE Transactions on Automatic Control.

[58]  A. Bandura,et al.  Negative self-efficacy and goal effects revisited. , 2003, The Journal of applied psychology.

[59]  W. Schultz.  Behavioral theories and the neurophysiology of reward. , 2006, Annual review of psychology.

[60]  Dianne C. Berry,et al.  The Role of Action in Implicit Learning , 1991 .

[61]  Michael R. Waldmann,et al.  Inferring interventional predictions from observational learning data , 2008, Psychonomic bulletin & review.

[62]  Dianne C. Berry,et al.  The combination of explicit and implicit learning processes in task control , 1987 .

[63]  P. Slovic,et al.  Analyzing use of diagnostic signs. , 1971, Investigative radiology.