A simple index rule for efficient traffic splitting over parallel wireless networks with partial information

Multi-path communication solutions provide a promising means to improve the network performance in areas covered by multiple wireless access networks. Today, little is known about how to effectively exploit this potential. We study a model where flows are transferred over multiple parallel networks, each of which is modeled as a processor sharing node. The goal is to minimize the expected transfer time of elastic data traffic by smartly dispatching the flows to the networks, based on partial information about the numbers of foreground and background flows in each of the nodes. In the case of full state information, the optimal policy can be derived via standard MDP-techniques, but for models with partial information an optimal solution is hard to obtain. An important requirement is that the splitting algorithm is efficient, yet simple, easy-to-implement, scalable in the number of parallel networks and robust against changes in the parameter settings. We propose a simple index rule for splitting traffic streams based on partial information, and benchmark the results against the optimal solution in the case of full state information. Extensive simulations with real networks show that this method performs extremely well under practical circumstances for a wide range of realistic parameter settings.

[1]  W. Lovejoy A survey of algorithmic methods for partially observed Markov decision processes , 1991 .

[2]  Ronen I. Brafman,et al.  A Heuristic Variable Grid Solution Method for POMDPs , 1997, AAAI/IAAI.

[3]  Pablo Rodriguez,et al.  Parallel-access for mirror sites in the Internet , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[4]  B. Sengupta,et al.  A conditional response time of the M/M/1 processor-sharing queue , 1985, AT&T Technical Journal.

[5]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[6]  S. Christian Albright,et al.  Structural Results for Partially Observable Markov Decision Processes , 1979, Oper. Res..

[7]  Robert D. van der Mei,et al.  Effective load for flow-level performance modelling of file transfers in wireless LANs , 2010, Comput. Commun..

[8]  Paramvir Bahl,et al.  MultiNet: connecting to multiple IEEE 802.11 networks using a single wireless card , 2004, IEEE INFOCOM 2004.

[9]  Sandjai Bhulai,et al.  Dynamic traffic splitting to parallel wireless networks with partial information: A Bayesian approach , 2012, Perform. Evaluation.

[10]  K. M. vanHee,et al.  Bayesian control of Markov chains , 1978 .

[11]  Milos Hauskrecht,et al.  Planning and control in stochastic domains with imperfect information , 1997 .

[12]  George E. Monahan,et al.  A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms , 2007 .

[13]  Tutomu Murase,et al.  Deployable multipath communication scheme with sufficient performance data distribution method , 2007, Comput. Commun..

[14]  E. M. Wright,et al.  Adaptive Control Processes: A Guided Tour , 1961, The Mathematical Gazette.

[15]  Robert D. van der Mei,et al.  Efficient traffic splitting in parallel TCP-based wireless networks: Modelling and experimental evaluation , 2013, Proceedings of the 2013 25th International Teletraffic Congress (ITC).

[16]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[17]  Raghupathy Sivakumar,et al.  A Transport Layer Approach for Achieving Aggregate Bandwidths on Multi-Homed Mobile Hosts , 2002, MobiCom '02.

[18]  A. Cassandra,et al.  Exact and approximate algorithms for partially observable markov decision processes , 1998 .

[19]  Sally Floyd,et al.  TCP Selective Acknowledgement Options , 1996 .

[20]  Carey L. Williamson,et al.  On processor sharing and its applications to cellular data network provisioning , 2007, Perform. Evaluation.

[21]  J. Duncanson,et al.  Inverse multiplexing , 1994, IEEE Communications Magazine.

[22]  Leslie Pack Kaelbling,et al.  Learning Policies for Partially Observable Environments: Scaling Up , 1997, ICML.

[23]  Donald C. Cox Fundamental limitations on increasing data rate in wireless systems , 2008, IEEE Communications Magazine.

[24]  G. Monahan State of the Art—A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms , 1982 .

[25]  Apostolos Burnetas,et al.  Optimal Adaptive Policies for Markov Decision Processes , 1997, Math. Oper. Res..

[26]  Patchigolla Kiran Kumar,et al.  A Survey of Some Results in Stochastic Adaptive Control , 1985 .

[27]  N. Hegde,et al.  Sojourn times in finite-capacity processor-sharing queues , 2005, Next Generation Internet Networks, 2005.

[28]  Chelsea C. White,et al.  A survey of solution techniques for the partially observed Markov decision process , 1991, Ann. Oper. Res..

[29]  Stuart J. Russell,et al.  Approximating Optimal Policies for Partially Observable Stochastic Domains , 1995, IJCAI.

[30]  Wenju Liu,et al.  Region-Based Approximations for Planning in Stochastic Domains , 1997, UAI.

[31]  Dilip Sarkar,et al.  Concurrent multipath transport , 2007, Comput. Commun..

[32]  John N. Tsitsiklis,et al.  The Complexity of Markov Decision Processes , 1987, Math. Oper. Res..

[33]  Mostafa Ammar,et al.  On the effect of large-scale deployment of parallel downloading , 2003, Proceedings the Third IEEE Workshop on Internet Applications. WIAPP 2003.

[34]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.