Some Classes of Imperfect Information Finite State-Space Stochastic Games with Finite-Dimensional Solutions

Abstract Stochastic games under imperfect information are typically computationally intractable even in the discrete-time/discrete-state case considered here. We consider a problem where one player has perfect information. A function of a conditional probability distribution is proposed as an information state. In the problem form here, the payoff is only a function of the terminal state of the system, and the initial information state is either linear or a sum of max-plus delta functions. When the initial information state belongs to these classes, its propagation is finite-dimensional. The state feedback value function is also finite-dimensional, and obtained via dynamic programming, but has a nonstandard form due to the necessity of an expanded state variable. Under a saddle point assumption, Certainty Equivalence is obtained and the proposed function is indeed an information state.

[1]  Marianne Akian,et al.  Densities of idempotent measures and large deviations , 1999 .

[2]  Wendell H. Fleming,et al.  Max-Plus Stochastic Processes , 2004 .

[3]  R. Elliott,et al.  The Existence Of Value In Differential Games , 1972 .

[4]  T. Başar,et al.  Dynamic Noncooperative Game Theory , 1982 .

[5]  John S. Baras,et al.  Partially Observed Differential Games, Infinite Dimensional HJI Equations, and Nonlinear HControl , 1994 .

[6]  Huihui Jiang,et al.  Modeling and control of military operations against adversarial control , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[7]  M. James,et al.  Robust H∞ output feedback control for nonlinear systems , 1995, IEEE Trans. Autom. Control..

[8]  W. Fleming,et al.  Optimal Control for Partially Observed Diffusions , 1982 .

[9]  C. Leake.  Synchronization and Linearity: An Algebra for Discrete Event Systems , 1994 .

[10]  H. S. Morse,et al.  The DARPA JFACC program: modeling and control of military operations , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[11]  J. Filar,et al.  Competitive Markov Decision Processes , 1996 .

[12]  Debasish Ghose,et al.  Game theoretic campaign modeling and analysis , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[13]  Pravin Varaiya,et al.  Stochastic Systems: Estimation, Identification, and Adaptive Control , 1986 .

[14]  William M. McEneaney.  A class of reasonably tractable partially observed discrete stochastic games , 2002, Proceedings of the 41st IEEE Conference on Decision and Control, 2002..

[15]  T. Basar,et al.  H∞-0ptimal Control and Related Minimax Design Problems: A Dynamic Game Approach , 1996, IEEE Trans. Autom. Control..

[16]  Willi Hock,et al.  Lecture Notes in Economics and Mathematical Systems , 1981 .

[17]  Mark L. Hanson,et al.  Mixed Initiative Planning and Control Under Uncertainty , 2002 .

[18]  K. Ito,et al.  Stochastic games and inverse Lyapunov methods in air operations , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[19]  William McEneaney,et al.  CONTROL FOR UAV OPERATIONS UNDER IMPERFECT INFORMATION , 2002 .

[20]  William M. McEneaney,et al.  A Max-Plus-Based Algorithm for a Hamilton--Jacobi--Bellman Equation of Nonlinear Filtering , 2000, SIAM J. Control. Optim..

[21]  W.M. McEneaney,et al.  Stochastic game approach to air operations , 2004, IEEE Transactions on Aerospace and Electronic Systems.

[22]  Milton B. Adams,et al.  Closed-Loop Hierarchical Control of Military Air Operations , 2002 .