The LP/POMDP marriage: Optimization with imperfect information

Anewtechniqueforsolvinglarge-scaleallocationproblemswithpartiallyobservable states and constrained action and observation resources is introduced. The technique uses a master linear program (LP) to determine allocations among a set of control policies, and uses partially observable Markov decision processes (POMDPs) to determine improving policies using dual prices from the master LP. An application is made to a military problem where aircraft attack targets in a sequence of stages, with information acquired in one stage being used to plan attacks in the next. c 2000 John Wiley & Sons, Inc. Naval Research Logistics 47: 607{619, 2000

[1]  Hsien-Te Cheng,et al.  Algorithms for partially observable markov decision processes , 1989 .

[2]  Kirk A. Yost.  Solution of large-scale allocation problems with partially observable outcomes , 1998 .

[3]  W. K. Haneveld.  Duality in Stochastic Linear and Dynamic Programming , 1986 .

[4]  D. Castañón.  Approximate dynamic programming for sensor management , 1997, Proceedings of the 36th IEEE Conference on Decision and Control.

[5]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Stochastic Control , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[6]  Ralph E. Gomory,et al.  A Linear Programming Approach to the Cutting Stock Problem---Part II , 1963 .

[7]  W. Lovejoy.  A survey of algorithmic methods for partially observed Markov decision processes , 1991 .

[8]  Georg Ch. Pflug.  On-Line Optimization of Simulated Markovian Processes , 1990, Math. Oper. Res..

[9]  Edward J. Sondik,et al.  The Optimal Control of Partially Observable Markov Processes over a Finite Horizon , 1973, Oper. Res..

[10]  E. J. Sondik,et al.  The Optimal Control of Partially Observable Markov Decision Processes. , 1971 .

[11]  R. Gomory,et al.  A Linear Programming Approach to the Cutting-Stock Problem , 1961 .

[12]  D. Bertsekas,et al.  Dynamic Programming and Stochastic Control , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[13]  A. Cassandra,et al.  Exact and approximate algorithms for partially observable markov decision processes , 1998 .

[14]  John N. Tsitsiklis,et al.  The Complexity of Markov Decision Processes , 1987, Math. Oper. Res..

[15]  David L. Woodruff,et al.  A class of stochastic programs withdecision dependent random elements , 1998, Ann. Oper. Res..

[16]  Kneale T. Marshall,et al.  Decision making and forecasting , 1995 .

[17]  Anthony R. Cassandra,et al.  Optimal Policies for Partially Observable Markov Decision Processes , 1994 .