A simple technique in Markovian control with applications to resource allocation in communication networks

The purpose of this note is to demonstrate that for semi-Markov decision problems with exponentially distributed transition times considerable computational improvements on the value-iteration algorithm can be obtained. This can be achieved by the simple trick of introducing fictitious decision epochs so that sparse transition matrices are created. Applications to optimal sharing of resources in communication networks are given.