Scalable constraint-based virtual data center allocation

Abstract Constraint-based techniques can solve challenging problems arising in highly diverse applications. This paper considers the problem of virtual data center (VDC) allocation, an important, emerging challenge for modern data center operators. To address this problem, we introduce Netsolver , a system for VDC allocation that is based on constraint solving. Netsolver represents a major improvement over existing approaches: it is sound, complete, and scalable, providing support for end-to-end, multi-path bandwidth guarantees across all the layers of hosting infrastructure, from servers to top-of-rack switches to aggregation switches to access routers. Netsolver scales to realistic data center sizes and VDC topologies, typically requiring just seconds to allocate VDCs of 5–15 virtual machines to physical data centers with 1000+ servers, maintaining this efficiency even when the data center is nearly saturated. In many cases, Netsolver can allocate 150 % − 300 % as many total VDCs to the same physical data center as previous methods. Finally, we show how Netsolver can be extended with additional optimization constraints, such as VM affinity and hotspot minimization, demonstrating the flexibility of our approach. The performance and flexibility of Netsolver are made possible by our formalization of the VDC allocation problem in terms of multi-commodity flows, and the corresponding efficient handling of network flow problems in the underlying constraint solvers. This shows the importance of supporting flow-based constraints, which are more mature in ILP- vs. SMT-based constraint solving.

[1]  Thomas Stützle,et al.  AClib: A Benchmark Library for Algorithm Configuration , 2014, LION.

[2]  Albert G. Greenberg,et al.  VL2: a scalable and flexible data center network , 2009, SIGCOMM '09.

[3]  Armin Biere,et al.  A survey of recent advances in SAT-based formal verification , 2005, International Journal on Software Tools for Technology Transfer.

[4]  Raouf Boutaba,et al.  Virtual Network Embedding with Coordinated Node and Link Mapping , 2009, IEEE INFOCOM 2009.

[5]  Daniel Kroening,et al.  SAT-Based Model Checking , 2018, Handbook of Model Checking.

[6]  Vasileios Pappas,et al.  Improving the Scalability of Data Center Networks with Traffic-aware Virtual Machine Placement , 2010, 2010 Proceedings IEEE INFOCOM.

[7]  Mark Handley,et al.  Improving datacenter performance and robustness with multipath TCP , 2011, SIGCOMM 2011.

[8]  Kevin Leyton-Brown,et al.  Sequential Model-Based Optimization for General Algorithm Configuration , 2011, LION.

[9]  Andrew V. Goldberg,et al.  Quincy: fair scheduling for distributed computing clusters , 2009, SOSP '09.

[10]  Holger Karl,et al.  A virtual network mapping algorithm based on subgraph isomorphism detection , 2009, VISA '09.

[11]  Scott Shenker,et al.  Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling , 2010, EuroSys '10.

[12]  Hitesh Ballani,et al.  End-to-end Performance Isolation Through Virtual Datacenters , 2014, OSDI.

[13]  Xiang Cheng,et al.  Virtual network embedding through topology-aware node ranking , 2011, CCRV.

[14]  Helen J. Wang,et al.  SecondNet: a data center network virtualization architecture with bandwidth guarantees , 2010, CoNEXT.

[15]  Asser N. Tantawi A Scalable Algorithm for Placement of Virtual Clusters in Large Data Centers , 2012, 2012 IEEE 20th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems.

[16]  David A. Maltz,et al.  Surviving failures in bandwidth-constrained datacenters , 2012, CCRV.

[17]  Raouf Boutaba,et al.  A multi-commodity flow based approach to virtual network resource allocation , 2003, GLOBECOM '03. IEEE Global Telecommunications Conference (IEEE Cat. No.03CH37489).

[18]  Amit Kumar,et al.  Provisioning a virtual private network: a network design problem for multicommodity flow , 2001, STOC '01.

[19]  Alan J. Hu,et al.  SAT Modulo Monotonic Theories , 2014, AAAI.

[20]  Gautam Kumar,et al.  FairCloud: sharing the network in cloud computing , 2011, CCRV.

[21]  Haitao Wu,et al.  BCube: a high performance, server-centric network architecture for modular data centers , 2009, SIGCOMM '09.

[22]  George Varghese,et al.  CONGA: distributed congestion-aware load balancing for datacenters , 2015, SIGCOMM.

[23]  Yazhe Tang,et al.  VirtualRack: Bandwidth-aware virtual network allocation for multi-tenant datacenters , 2014, 2014 IEEE International Conference on Communications (ICC).

[24]  Albert G. Greenberg,et al.  A flexible model for resource management in virtual private networks , 1999, SIGCOMM '99.

[25]  Vyas Sekar,et al.  Making middleboxes someone else's problem: network processing as a cloud service , 2012, SIGCOMM '12.

[26]  Lisandro Zambenedetti Granville,et al.  Data Center Network Virtualization: A Survey , 2013, IEEE Communications Surveys & Tutorials.

[27]  Dawson R. Engler,et al.  KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs , 2008, OSDI.

[28]  Hitesh Ballani,et al.  Towards predictable datacenter networks , 2011, SIGCOMM 2011.

[29]  Mustafa Y. Sir,et al.  An optimization-based approach for connecting partitioned mobile sensor/Actuator Networks , 2011, 2011 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[30]  Gwendal Simon,et al.  VDC Planner: Dynamic migration-aware Virtual Data Center embedding for clouds , 2013, 2013 IFIP/IEEE International Symposium on Integrated Network Management (IM 2013).

[31]  Sujata Banerjee,et al.  Application-driven bandwidth guarantees in datacenters , 2015, SIGCOMM.

[32]  Alon Itai,et al.  On the complexity of time table and multi-commodity flow problems , 1975, 16th Annual Symposium on Foundations of Computer Science (sfcs 1975).

[33]  Matthias Rost,et al.  Beyond the Stars: Revisiting Virtual Cluster Embeddings , 2015, CCRV.

[34]  Minlan Yu,et al.  Rethinking virtual network embedding: substrate support for path splitting and migration , 2008, CCRV.

[35]  Vasudeva Varma,et al.  Network-aware virtual machine consolidation for large data centers , 2013, NDM '13.

[36]  Ahmed Karmouch,et al.  Resource Discovery and Allocation in Network Virtualization , 2012, IEEE Communications Surveys & Tutorials.

[37]  Reuven Bar-Yehuda,et al.  A Local-Ratio Theorem for Approximating the Weighted Vertex Cover Problem , 1983, WG.

[38]  Ivan Beschastnikh,et al.  Scalable Constraint-based Virtual Data Center Allocation , 2017, IJCAI.

[39]  Xavier Hesselbach,et al.  ALEVIN - A Framework to Develop, Compare, and Analyze Virtual Network Embedding Algorithms , 2011, Electron. Commun. Eur. Assoc. Softw. Sci. Technol..

[40]  Xavier Hesselbach,et al.  Virtual Network Embedding: A Survey , 2013, IEEE Communications Surveys & Tutorials.

[41]  Nikolaj Bjørner,et al.  Z3: An Efficient SMT Solver , 2008, TACAS.

[42]  Yifei Yuan,et al.  On the feasibility of automation for bandwidth allocation problems in data centers , 2013, 2013 Formal Methods in Computer-Aided Design.

[43]  Robert L. Smith,et al.  A MIXED INTEGER LINEAR PROGRAMMING MODEL FOR DYNAMIC ROUTE GUIDANCE , 1998 .

[44]  Niklas Sörensson,et al.  Translating Pseudo-Boolean Constraints into SAT , 2006, J. Satisf. Boolean Model. Comput..

[45]  Laurence A. Wolsey,et al.  Cutting planes in integer and mixed integer programming , 2002, Discret. Appl. Math..