Linear programming and continuous markovian decision problems

Summary This paper is concerned with a continuous time parameter Markovian sequential decision process, and presents a method which transforms a given continuous parameter problem into a discrete one. It is proved that the optimal stationary policy for the resulting discrete time parameter Markovian decision process is also the optimal stationary policy for the original continuous one, and vice versa. The resulting discrete parameter problem may be more easily solved than the continuous one by applying the linear programming method. A simple numerical example is presented.