Bias Based General Framework for Delay Reduction in Backpressure Routing Algorithm

In queueing networks, it is well known that the throughput-optimal backpressure routing algorithm results in poor delay performance for light and moderate traffic loads. To improve delay performance of backpressure routing algorithm, available works exploit various information of queueing networks, such as queue length, shortest path and packet delay, to direct packets to shorter routes to their destinations. Despite different forms of these works, they share the common characteristic: using bias to help backpressure routing to reduce packet delay. From this observation, we propose in this paper a bias based general framework to reduce packet delay of backpressure routing algorithm. Our framework is general in the sense that it not only covers many bias based variants of backpressure routing algorithm as special cases but also enables machine learning based methods to be adopted to further improve delay performance. Based on the general framework, we propose a specific Q-learning based backpressure routing algorithm to reduce packet delay.