A neuro-dynamic programming approach to call admission control in integrated service networks : the single link case

We formulate the call admission control problem for a single link in an integrated service environment as a Markov Decision Problem. In principle, an optimal admission control policy can be computed using methods of Dynamic Programming. However, as the number of possible states of the underlying Markov Chain grows exponentially in the number of customer classes, Dynamic Programming algorithms for realistic size problems are computationally infeasible. We try to overcome this so-called "curse of dimensionality" by using methods of Neuro-Dynamic Programming (NDP for short). NDP employs simulation-based algorithms and function approximation techniques to find control policies for large-scale Markov Decision Problems. We apply two methods of NDP to the call admission control problem: the TD(O) algorithm and Approximate Policy Iteration. We assess the performance of these methods by comparing with two heuristic policies: a policy which always accepts a new customer when the required resources are available, and a threshold policy. 1 This research was supported by a contract with Siemens AG, Munich, Germany.