An Alternating Direction Method Approach to Cloud Traffic Management

In this paper, we introduce a unified framework for studying various cloud traffic management problems, ranging from geographical load balancing to backbone traffic engineering. We first abstract these real-world problems as a multi-facility resource allocation problem, and then present two distributed optimization algorithms by exploiting the special structure of the problem. Our algorithms are inspired by Alternating Direction Method of Multipliers (ADMM), enjoying a number of unique features. Compared to dual decomposition, they converge with non-strictly convex objective functions; compared to other ADMM-type algorithms, they not only achieve faster convergence under weaker assumptions, but also have lower computational complexity and lower message-passing overhead. The simulation results not only confirm these desirable features of our algorithms, but also highlight several additional advantages, such as scalability and fault-tolerance.

[1]  Jack Dongarra,et al.  MPI: The Complete Reference , 1996 .

[2]  Dimitri P. Bertsekas,et al.  Network optimization : continuous and discrete models , 1998 .

[3]  Bingsheng He,et al.  On non-ergodic convergence rate of Douglas–Rachford alternating direction method of multipliers , 2014, Numerische Mathematik.

[4]  Baochun Li,et al.  Joint request mapping and response routing for geo-distributed cloud services , 2013, 2013 Proceedings IEEE INFOCOM.

[5]  Stephen P. Boyd,et al.  Subgradient Methods , 2007 .

[6]  Rakesh V. Vohra,et al.  Mathematics of the Internet , 2001 .

[7]  Ramesh Govindan,et al.  Trumpet: Timely and Precise Triggers in Data Centers , 2016, SIGCOMM.

[8]  J. Rexford,et al.  Scalable Multi-Class Traffic Management in Data Center Backbone Networks , 2013, IEEE Journal on Selected Areas in Communications.

[9]  Ron Kohavi,et al.  Practical guide to controlled experiments on the web: listen to your customers not to the hippo , 2007, KDD '07.

[10]  Adam Wierman,et al.  Renewable and cooling aware workload management for sustainable data centers , 2012, SIGMETRICS '12.

[11]  Albert G. Greenberg,et al.  Optimizing Cost and Performance in Online Service Provider Networks , 2010, NSDI.

[12]  Mung Chiang,et al.  Distributed wide-area traffic management for cloud services , 2012, SIGMETRICS '12.

[13]  Bingsheng He,et al.  On the O(1/n) Convergence Rate of the Douglas-Rachford Alternating Direction Method , 2012, SIAM J. Numer. Anal..

[14]  Devavrat Shah,et al.  Flowtune: Flowlet Control for Datacenter Networks , 2017, NSDI.

[15]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[16]  Zhi-Quan Luo,et al.  Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems , 2015, ICASSP.

[17]  Nick Feamster,et al.  Design and implementation of a routing control platform , 2005, NSDI.

[18]  Bingsheng He,et al.  The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent , 2014, Mathematical Programming.

[19]  Chase Cotton,et al.  Packet-level traffic measurements from the Sprint IP backbone , 2003, IEEE Netw..

[20]  Xiaoming Yuan,et al.  A Note on the Alternating Direction Method of Multipliers , 2012, J. Optim. Theory Appl..

[21]  A. Robert Calderbank,et al.  Layering as Optimization Decomposition: A Mathematical Theory of Network Architectures , 2007, Proceedings of the IEEE.

[22]  Arjun Singh,et al.  A practical algorithm for balancing the max-min fairness and throughput objectives in traffic engineering , 2012, 2012 Proceedings IEEE INFOCOM.

[23]  Shiqian Ma,et al.  Solving Multiple-Block Separable Convex Minimization Problems Using Two-Block Alternating Direction Method of Multipliers , 2013, ArXiv.

[24]  Steven H. Low,et al.  A duality model of TCP and queue management algorithms , 2003, TNET.

[25]  Baochun Li,et al.  Temperature Aware Workload Managementin Geo-Distributed Data Centers , 2013, IEEE Transactions on Parallel and Distributed Systems.

[26]  Srinivasan Keshav,et al.  It's not easy being green , 2012, CCRV.

[27]  Murali S. Kodialam,et al.  Traffic engineering in software defined networks , 2013, 2013 Proceedings IEEE INFOCOM.

[28]  John F. Canny,et al.  Butterfly Mixing: Accelerating Incremental-Update Algorithms on Clusters , 2013, SDM.

[29]  Bruce M. Maggs,et al.  Cutting the electric bill for internet-scale systems , 2009, SIGCOMM '09.

[30]  Frank Kelly,et al.  Rate control for communication networks: shadow prices, proportional fairness and stability , 1998, J. Oper. Res. Soc..

[31]  Xu Li,et al.  Min Flow Rate Maximization for Software Defined Radio Access Networks , 2013, IEEE Journal on Selected Areas in Communications.

[32]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[33]  Steven H. Low,et al.  Optimization flow control—I: basic algorithm and convergence , 1999, TNET.

[34]  Srikanth Kandula,et al.  Achieving high utilization with software-driven WAN , 2013, SIGCOMM.

[35]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[36]  Derong Liu.  The Mathematics of Internet Congestion Control , 2005, IEEE Transactions on Automatic Control.

[37]  Min Zhu,et al.  B4: experience with a globally-deployed software defined wan , 2013, SIGCOMM.

[38]  B. Chapman.  Federal Energy Regulatory Commission, U.S. , 2013 .

[39]  Zhi-Quan Luo,et al.  On the linear convergence of the alternating direction method of multipliers , 2012, Mathematical Programming.

[40]  Jean C. Walrand,et al.  Fair end-to-end window-based congestion control , 2000, TNET.

[41]  Wolf-Dietrich Weber,et al.  Power provisioning for a warehouse-sized computer , 2007, ISCA '07.

[42]  Wotao Yin,et al.  On the Global and Linear Convergence of the Generalized Alternating Direction Method of Multipliers , 2016, J. Sci. Comput..

[43]  Cheng Jin,et al.  MATE: MPLS adaptive traffic engineering , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[44]  M. Hestenes.  Multiplier and gradient methods , 1969 .

[45]  Ramesh K. Sitaraman,et al.  The Akamai network: a platform for high-performance internet applications , 2010, OPSR.

[46]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[47]  Ness B. Shroff,et al.  Utility maximization for communication networks with multipath routing , 2006, IEEE Transactions on Automatic Control.

[48]  Shiqian Ma,et al.  On the Global Linear Convergence of the ADMM with MultiBlock Variables , 2014, SIAM J. Optim..

[49]  Daniel Pérez Palomar,et al.  A tutorial on decomposition methods for network utility maximization , 2006, IEEE Journal on Selected Areas in Communications.

[50]  Hong Yan,et al.  A clean slate 4D approach to network control and management , 2005, CCRV.

[51]  A. Rowstron,et al.  Towards predictable datacenter networks , 2011, SIGCOMM.

[52]  Bo Li,et al.  Fuel Cell Generation in Geo-Distributed Cloud Services: A Quantitative Study , 2014, 2014 IEEE 34th International Conference on Distributed Computing Systems.

[53]  Lachlan L. H. Andrew,et al.  Greening Geographical Load Balancing , 2015, IEEE/ACM Transactions on Networking.