A Tractable Leader-Follower MDP Model for Animal Disease Management

Sustainable animal disease management requires to design and implement control policies at the regional scale. However, for diseases which are not regulated, individual farmers are responsible for the adoption and successful application of control policies at the farm scale. Organizations (groups of farmers, health institutions...) may try to influence farmers' control actions through financial incentives, in order to ensure sustainable (from the health and economical point of views) disease management policies. Economics / Operations Research frameworks have been proposed for modeling the effect of incentives on agents. The Leader-Follower Markov Decision Processes framework is one such framework, that combines Markov Decision Processes (MDP) and stochastic games frameworks. However, since finding equilibrium policies in stochastic games is hard when the number of players is large, LF-MDP problems are intractable. Our contribution, in this article, is to propose a tractable model of the animal disease management problem. The tractable model is obtained through a few simple modeling approximations which are acceptable when the problem is viewed from the organization side. As a result, we design a polynomial-time algorithm for animal disease management, which we evaluate on a case study inspired from the problem of controlling the spread of the Porcine Reproductive and Respiratory Syndrome (PRRS).

[1]  Aranyak Mehta,et al.  Playing large games using simple strategies , 2003, EC '03.

[2]  Craig Boutilier,et al.  Coordination in multiagent reinforcement learning: a Bayesian approach , 2003, AAMAS '03.

[3]  Nathalie Peyrard,et al.  A framework and a mean-field algorithm for the local control of spatial processes , 2012, Int. J. Approx. Reason..

[4]  Xiaotie Deng,et al.  Settling the complexity of computing two-player Nash equilibria , 2007, JACM.

[5]  O. Mangasarian,et al.  Two-person nonzero-sum games and quadratic programming , 1964 .

[6]  H. Kuk On equilibrium points in bimatrix games , 1996 .

[7]  G. Nodelijk,et al.  Porcine Reproductive and Respiratory Syndrome (PRRS) with special reference to clinical aspects and diagnosis: A review , 2002, The Veterinary quarterly.

[8]  Paul W. Goldberg,et al.  The complexity of computing a Nash equilibrium , 2006, STOC '06.

[9]  J. Filar,et al.  Competitive Markov Decision Processes , 1996 .

[10]  Siddhartha Bhattacharyya,et al.  Single-leader–multiple-follower games with boundedly rational agents , 2009 .

[11]  Erica L. Plambeck,et al.  Performance-Based Incentives in a Dynamic Principal-Agent Model , 2000, Manuf. Serv. Oper. Manag..

[12]  Michael P. Wellman,et al.  Nash Q-Learning for General-Sum Stochastic Games , 2003, J. Mach. Learn. Res..

[13]  S. Bhattacharyya,et al.  Leader-Follower semi-Markov Decision Problems: Theoretical Framework and Approximate Solution , 2007, 2007 IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning.

[14]  Herbert W. Hethcote,et al.  The Mathematics of Infectious Diseases , 2000, SIAM Rev..