Deterministic Modeling and Evaluation of Decision-Making Dynamics in Sequential Two-Alternative Forced Choice Tasks

The focus of the work in this paper is a systems-theoretic construction, analysis, and evaluation of a deterministic model of human decision making relative to experimental data. In sequential two-alternative forced choice decision tasks, a human subject is presented with two choices at every time step, is given finite time to select one of the choices, and receives a reward after a choice is made (presented as a number on a computer screen). The goal for the human is to obtain the maximal reward while not knowing the underlying reward assignment process. In this work, we present a parameterized deterministic model for human decision making in this context and analyze optimality and stability using a finite state machine approach. This model is then evaluated relative to experimental data from human subjects performing each of six tasks.

[1]  J. Gregory Trafton,et al.  Human control of multiple unmanned vehicles: effects of interface type on execution and task switching times , 2006, HRI '06.

[2]  Frederik W. Heger,et al.  Human-Robot Teams for Large-Scale Assembly , 2007 .

[3]  Philip Holmes,et al.  A simple decision task in a social context: Experiments, a model, and preliminary analyses of behavioral data , 2008, 2008 47th IEEE Conference on Decision and Control.

[4]  W. Marsden.  I and J , 2012 .

[5]  R. Herrnstein.  Rational Choice Theory Necessary but Not Sufficient , 1990 .

[6]  Jonathan D. Cohen,et al.  The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. , 2006, Psychological review.

[7]  Kristi A. Morgansen,et al.  Modeling and analysis of dynamic decision making in sequential two-choice tasks , 2008, 2008 47th IEEE Conference on Decision and Control.

[8]  Adele Diederich,et al.  Simple matrix methods for analyzing diffusion models of choice probability, choice response time, and simple response time , 2003 .

[9]  Kristi A. Morgansen,et al.  Modeling and evaluation of decision-making dynamics in sequential two-alternative forced choice tasks , 2010, 49th IEEE Conference on Decision and Control (CDC).

[10]  John P. Lewis,et al.  The Future of Disaster Response: Humans Working with Multiagent Teams using DEFACTO , 2005, AAAI Spring Symposium: AI Technologies for Homeland Security.

[11]  Philip L. Smith,et al.  Stochastic Dynamic Models of Response Time and Accuracy: A Foundational Primer. , 2000, Journal of mathematical psychology.

[12]  Harold H. Kelley,et al.  The development of cooperation in the "minimal social situation". , 1962 .

[13]  Naomi Ehrich Leonard,et al.  Integrating human and robot decision-making dynamics with feedback: Models and convergence analysis , 2008, 2008 47th IEEE Conference on Decision and Control.

[14]  Naomi Ehrich Leonard,et al.  Steady-state distributions for human decisions in two-alternative choice tasks , 2010, Proceedings of the 2010 American Control Conference.

[15]  B. Trouvain,et al.  Evaluation of multi-robot control and monitoring performance , 2002, Proceedings. 11th IEEE International Workshop on Robot and Human Interactive Communication.

[16]  Raja Parasuraman,et al.  Adaptive Automation for Human-Robot Teaming in Future Command and Control Systems , 2007 .

[17]  P. Montague,et al.  A Computational Role for Dopamine Delivery in Human Decision-Making , 1998, Journal of Cognitive Neuroscience.

[18]  Heath A. Ruff,et al.  EXPLORING AUTOMATION ISSUES IN SUPERVISORY CONTROL OF MULTIPLE UAVS , 2004 .

[19]  Angela J. Yu,et al.  Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[20]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[21]  Philip Holmes,et al.  A Decision Task in a Social Context: Human Experiments, Models, and Analyses of Behavioral Data , 2012, Proceedings of the IEEE.

[22]  J. Townsend,et al.  Fundamental derivations from decision field theory , 1992 .

[23]  Naomi Ehrich Leonard,et al.  Convergence in human decision-making dynamics , 2010, Syst. Control. Lett..

[24]  Christopher D. Wickens,et al.  Unmanned Aerial Vehicle Flight Control: False Alarms versus Misses , 2004 .

[25]  P. Montague,et al.  Neural Economics and the Biological Substrates of Valuation , 2002, Neuron.

[26]  H. Robbins.  Some aspects of the sequential design of experiments , 1952 .

[27]  Naomi Ehrich Leonard,et al.  Towards Human–Robot Teams: Model-Based Analysis of Human Decision Making in Two-Alternative Choice Tasks With Social Feedback , 2012, Proceedings of the IEEE.

[28]  J R Beck,et al.  Markov Models in Medical Decision Making , 1993, Medical decision making : an international journal of the Society for Medical Decision Making.

[29]  Diederich,et al.  Dynamic Stochastic Models for Decision Making under Time Constraints , 1997, Journal of mathematical psychology.

[30]  Carl Nehme,et al.  Modeling human supervisory control in heterogeneous unmanned vehicle systems , 2009 .

[31]  John Baillieul,et al.  Search decisions for teams of automata , 2008, 2008 47th IEEE Conference on Decision and Control.