Prefrontal cortex and decision making in a mixed-strategy game

In a multi-agent environment, where the outcomes of one's actions change dynamically because they are related to the behavior of other beings, it becomes difficult to make an optimal decision about how to act. Although game theory provides normative solutions for decision making in groups, how such decision-making strategies are altered by experience is poorly understood. These adaptive processes might resemble reinforcement learning algorithms, which provide a general framework for finding optimal strategies in a dynamic environment. Here we investigated the role of prefrontal cortex (PFC) in dynamic decision making in monkeys. As in reinforcement learning, the animal's choice during a competitive game was biased by its choice and reward history, as well as by the strategies of its opponent. Furthermore, neurons in the dorsolateral prefrontal cortex (DLPFC) encoded the animal's past decisions and payoffs, as well as the conjunction between the two, providing signals necessary to update the estimates of expected reward. Thus, PFC might have a key role in optimizing decision-making strategies.

[1]  E. Rowland.  Theory of Games and Economic Behavior , 1946, Nature.

[2]  J. Nash.  Equilibrium Points in N-Person Games. , 1950, Proceedings of the National Academy of Sciences of the United States of America.

[3]  J. Robinson.  AN ITERATIVE METHOD OF SOLVING A GAME , 1951, Classics in Game Theory.

[4]  H. Simon,et al.  Models of Man. , 1957 .

[5]  J. Touhey.  Decision Processes, Expectations, and Adoption of Strategies in Zero-Sum Games , 1974 .

[6]  Elizabeth C. Hirschman,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[7]  Rüdeger Baumann,et al.  Games of Strategy , 1982 .

[8]  C. Bruce,et al.  Primate frontal eye fields. II. Physiological and anatomical correlates of electrically evoked eye movements. , 1985, Journal of neurophysiology.

[9]  P. Kline.  Models of man , 1986, Nature.

[10]  B. O'Neill.  Nonmetric test of the minimax theory of two-person zerosum games. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[11]  R. Byrne,et al.  Machiavellian intelligence : social expertise and the evolution of intellect in monkeys, apes, and humans , 1990 .

[12]  Colin Camerer.  Behavioral Game Theory , 1990 .

[13]  Robert W. Rosenthal,et al.  Testing the Minimax Hypothesis: A Re-examination of O'Neill's Game Experiment , 1990 .

[14]  Barry O'Neill,et al.  Comments on Brown and Rosenthal's Reexamination [Testing the Minimax Hypothesis, A Reexamination of O'Neill's Game Experiment] , 1991 .

[15]  A. Rapoport,et al.  Generation of random series in two-person strictly competitive games , 1992 .

[16]  A. Rapoport,et al.  Mixed strategies in strictly competitive games: A further test of the minimax hypothesis , 1992 .

[17]  A. Rapoport,et al.  Subjective randomization in one-and two-person games , 1994 .

[18]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[19]  Dilip Mookherjee,et al.  Learning behavior in an experimental matching pennies game , 1994 .

[20]  J. Ochs.  Games with Unique, Mixed Strategy Equilibria: An Experimental Study , 1995 .

[21]  Masataka Watanabe.  Reward expectancy in primate prefrental neurons , 1996, Nature.

[22]  Ronald Christensen,et al.  Log-Linear Models and Logistic Regression , 1997 .

[23]  A. Rubinstein.  Modeling Bounded Rationality , 1998 .

[24]  R. Byrne,et al.  Machiavellian intelligence II : extensions and evaluations , 1997 .

[25]  A. Rapoport,et al.  Randomization in individual choice behavior. , 1997 .

[26]  J. Friedman,et al.  Boundedly Rational Nash Equilibrium: A Probabilistic Choice Approach☆☆☆ , 1997 .

[27]  A. Roth,et al.  Predicting How People Play Games: Reinforcement Learning in Experimental Games with Unique, Mixed Strategy Equilibria , 1998 .

[28]  David R. Anderson,et al.  Model Selection and Multimodel Inference , 2003 .

[29]  O. Hikosaka,et al.  Expectation of reward modulates cognitive signals in the basal ganglia , 1998, Nature Neuroscience.

[30]  D. Fudenberg,et al.  The Theory of Learning in Games , 1998 .

[31]  M. Shadlen,et al.  Effect of Expected Reward Magnitude on the Response of Neurons in the Dorsolateral Prefrontal Cortex of the Macaque , 1999, Neuron.

[32]  Michael L. Platt,et al.  Neural correlates of decision variables in parietal cortex , 1999, Nature.

[33]  M. Posner.  The Brain and Emotion , 1999, Nature Medicine.

[34]  J. Schall,et al.  Performance monitoring by the supplementary eye ® eld , 2000 .

[35]  J Schlag,et al.  Reward-predicting and reward-detecting neuronal activity in the primate supplementary eye field. , 2000, Journal of neurophysiology.

[36]  K. Binmore,et al.  Does Minimax Work? An Experimental Study , 2001 .

[37]  J. Wooders,et al.  Minimax Play at Wimbledon , 2001 .

[38]  Farshid Vahid,et al.  Predicting How People Play Games: A Simple Dynamic Model of Choice , 2001, Games Econ. Behav..

[39]  O. Hikosaka,et al.  Influence of reward expectation on visuospatial processing in macaque lateral prefrontal cortex. , 2002, Journal of neurophysiology.

[40]  Xiao-Jing Wang,et al.  Probabilistic Decision Making by Slow Reverberation in Cortical Circuits , 2002, Neuron.

[41]  Jason M. Shachat,et al.  Mixed Strategy Play and the Minimax Hypothesis , 2002, J. Econ. Theory.

[42]  B. Richmond,et al.  Anterior Cingulate: Single Neuronal Signals Related to Degree of Reward Expectancy , 2002, Science.

[43]  P. Glimcher.  decisions, uncertainty and the brain , 2003 .

[44]  M. Roesch,et al.  Impact of expected reward on neuronal activity in prefrontal cortex, frontal and supplementary eye fields and premotor cortex. , 2003, Journal of neurophysiology.

[45]  Okihide Hikosaka,et al.  Reward-Dependent Gain and Bias of Visual Responses in Primate Superior Colliculus , 2003, Neuron.

[46]  J. C. Crowley,et al.  Saccade Reward Signals in Posterior Cingulate Cortex , 2003, Neuron.

[47]  Joshua W. Brown,et al.  Performance Monitoring by the Anterior Cingulate Cortex During Saccade Countermanding , 2003, Science.

[48]  S. Tsujimoto,et al.  Neuronal representation of response-outcome in the primate prefrontal cortex. , 2004, Cerebral cortex.

[49]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 2005, IEEE Transactions on Neural Networks.