Game-Theoretic Question Selection for Tests

Conventionally, the questions on a test are assumed to be kept secret from test takers until the test. However, for tests that are taken on a large scale, particularly asynchronously, this is very hard to achieve. For example, example TOEFL iBT and driver's license test questions are easily found online. This also appears likely to become an issue for Massive Open Online Courses (MOOCs). In this paper, we take the loss of confidentiality as a fact. Even so, not all hope is lost as the test taker can memorize only a limited set of questions' answers, and the tester can randomize which questions appear on the test. We model this as a Stackelberg game, where the tester commits to a mixed strategy and the follower responds. We provide an exponential-size linear program formulation, prove several NP-hardness results, and give efficient algorithms for special cases.

[1]  Vincent Conitzer,et al.  New complexity results about Nash equilibria , 2008, Games Econ. Behav..

[2]  Xiaotie Deng,et al.  Settling the complexity of computing two-player Nash equilibria , 2007, JACM.

[3]  Manish Jain,et al.  Computing optimal randomized resource allocations for massive security games , 2009, AAMAS 2009.

[4]  Manish Jain,et al.  Computing optimal randomized resource allocations for massive security games , 2009, AAMAS.

[5]  Bernhard von Stengel,et al.  Leadership games with convex strategy sets , 2010, Games Econ. Behav..

[6]  Robert J. Vanderbei,et al.  Linear Programming: Foundations and Extensions , 1998, Kluwer international series in operations research and management service.

[7]  J. Neumann.  Zur Theorie der Gesellschaftsspiele , 1928 .

[8]  Vincent Conitzer,et al.  Computing the optimal strategy to commit to , 2006, EC '06.

[9]  Martin Grötschel,et al.  The ellipsoid method and its consequences in combinatorial optimization , 1981, Comb..

[10]  Bo An,et al.  PROTECT: a deployed game theoretic system to protect the ports of the United States , 2012, AAMAS.

[11]  Richard M. Karp,et al.  Theoretical Improvements in Algorithmic Efficiency for Network Flow Problems , 1972, Combinatorial Optimization.

[12]  Sarit Kraus,et al.  Playing games for security: an efficient exact algorithm for solving Bayesian Stackelberg games , 2008, AAMAS.

[13]  Eitan Zemel,et al.  Nash and correlated equilibria: Some complexity considerations , 1989 .

[14]  Dan Suciu,et al.  Journal of the ACM , 2006 .

[15]  Vincent Conitzer,et al.  Complexity of Computing Optimal Stackelberg Strategies in Security Resource Allocation Games , 2010, AAAI.

[16]  Vincent Conitzer,et al.  Learning and Approximating the Optimal Strategy to Commit To , 2009, SAGT.

[17]  ChenXi,et al.  Settling the complexity of computing two-player Nash equilibria , 2009 .

[18]  Oriol Carbonell-Nicolau.  Games and Economic Behavior , 2011 .

[19]  Sarit Kraus,et al.  Bayesian stackelberg games and their application for security at Los Angeles international airport , 2008, SECO.

[20]  Vincent Conitzer,et al.  Solving Zero-Sum Security Games in Discretized Spatio-Temporal Domains , 2014, AAAI.

[21]  Vincent Conitzer,et al.  Stackelberg vs. Nash in Security Games: An Extended Investigation of Interchangeability, Equivalence, and Uniqueness , 2011, J. Artif. Intell. Res..

[22]  K. Hew,et al.  Students’ and instructors’ use of massive open online courses (MOOCs): Motivations and challenges , 2014 .

[23]  L. G. H. Cijan.  A polynomial algorithm in linear programming , 1979 .

[24]  Vincent Conitzer,et al.  Solving Security Games on Graphs via Marginal Probabilities , 2013, AAAI.

[25]  Milind Tambe,et al.  TRUSTS: Scheduling Randomized Patrols for Fare Inspection in Transit Systems , 2012, IAAI.

[26]  L. Khachiyan.  Polynomial algorithms in linear programming , 1980 .

[27]  C. Han.  A fast algorithm for the minimax flow problem with 01 weights , 1997 .

[28]  Sarit Kraus,et al.  Using Game Theory for Los Angeles Airport Security , 2009, AI Mag..

[29]  Justin Reich,et al.  Rebooting MOOC Research , 2015, Science.

[30]  Paul W. Goldberg,et al.  The Complexity of Computing a Nash Equilibrium , 2009, SIAM J. Comput..

[31]  Manish Jain,et al.  Quality-bounded solutions for finite Bayesian Stackelberg games: scaling up , 2011, AAMAS.

[32]  Steve Cooper,et al.  Reflections on Stanford's MOOCs , 2013, CACM.

[33]  H PapadimitriouChristos,et al.  The complexity of computing a Nash equilibrium , 2009 .

[34]  Milind Tambe,et al.  Protecting Moving Targets with Multiple Mobile Resources , 2013, J. Artif. Intell. Res..

[35]  Milind Tambe,et al.  Security and Game Theory: IRIS – A Tool for Strategic Security Allocation in Transportation Networks , 2011, AAMAS 2011.

[36]  Jean-Pierre Bourguignon,et al.  Mathematische Annalen , 1893 .

[37]  Edmund H. Durfee,et al.  Coherent Cooperation Among Communicating Problem Solvers , 1987, IEEE Transactions on Computers.