NetDEO: Automating network design, evolution, and optimization

With the ever-increasing number and complexity of applications deployed in data centers, the underlying network infrastructure can no longer sustain such a trend and exhibits several problems, such as resource fragmentation and low bisection bandwidth. In pursuit of a real-world applicable data center network (DCN) optimization approach that continuously maintains balanced network performance with high cost effectiveness, we design a topology independent resource allocation and optimization approach, NetDEO. Based on a swarm intelligence optimization model, NetDEO improves the scalability of the DCN by relocating virtual machines (VMs) and matching resource demand and availability. NetDEO is capable of (1) incrementally optimizing an existing VM placement in a data center; (2) deriving optimal deployment plans for newly added VMs; and (3) providing hardware upgrade suggestions and allowing the DCN to evolve as the workload changes over time. We evaluate the performance of NetDEO using realistic workload traces and simulated large-scale DCN under various topologies.

[1]  Teofilo F. Gonzalez,et al.  P-Complete Approximation Problems , 1976, J. ACM.

[2]  Aameek Singh,et al.  Coupled placement in modern data centers , 2009, 2009 IEEE International Symposium on Parallel & Distributed Processing.

[3]  Haitao Wu,et al.  FiConn: Using Backup Port for Server Interconnection in Data Centers , 2009, IEEE INFOCOM 2009.

[4]  Amin Vahdat,et al.  Service Placement in a Shared Wide-Area Platform , 2006, USENIX Annual Technical Conference, General Track.

[5]  Xiaoyun Zhu,et al.  Resource Assignment for Large-Scale Computing Utilities using Mathematical Programming , 2003 .

[6]  Haitao Wu,et al.  BCube: a high performance, server-centric network architecture for modular data centers , 2009, SIGCOMM '09.

[7]  Hitesh Ballani,et al.  Towards predictable datacenter networks , 2011, SIGCOMM 2011.

[8]  Paramvir Bahl,et al.  Flyways To De-Congest Data Center Networks , 2009, HotNets.

[9]  Madeleine Glick,et al.  Your Data Center Is a Router: The Case for Reconfigurable Optical Circuit Switched Paths , 2009, HotNets.

[10]  Jeffrey C. Mogul,et al.  SPAIN: COTS Data-Center Ethernet for Multipathing over Arbitrary Topologies , 2010, NSDI.

[11]  Konstantina Papagiannaki,et al.  c-Through: part-time optics in data centers , 2010, SIGCOMM 2010.

[12]  Emile H. L. Aarts,et al.  Simulated Annealing: Theory and Applications , 1987, Mathematics and Its Applications.

[13]  Lei Shi,et al.  Dcell: a scalable and fault-tolerant network structure for data centers , 2008, SIGCOMM '08.

[14]  Bruce E. Hajek,et al.  The time complexity of maximum matching by simulated annealing , 1988, JACM.

[15]  Helen J. Wang,et al.  SecondNet: a data center network virtualization architecture with bandwidth guarantees , 2010, CoNEXT.

[16]  Amin Vahdat,et al.  A scalable, commodity data center network architecture , 2008, SIGCOMM '08.

[17]  Haifeng Chen,et al.  VCAE: A Virtualization and Consolidation Analysis Engine for Large Scale Data Centers , 2010, SASO.

[18]  Vasileios Pappas,et al.  Improving the Scalability of Data Center Networks with Traffic-aware Virtual Machine Placement , 2010, 2010 Proceedings IEEE INFOCOM.

[19]  Abhishek Chandra,et al.  Starling: Minimizing Communication Overhead in Virtualized Computing Platforms Using Decentralized Affinity-Aware Migration , 2010, 2010 39th International Conference on Parallel Processing.

[20]  David Patterson,et al.  Service placement in shared wide-area platforms , 2005, SOSP '05.

[21]  Marco Dorigo,et al.  Swarm intelligence: from natural to artificial systems , 1999 .

[22]  Refael Hassin,et al.  Approximating the minimum quadratic assignment problems , 2009, TALG.

[23]  Amin Vahdat,et al.  Helios: a hybrid electrical/optical switch architecture for modular data centers , 2010, SIGCOMM '10.

[24]  Atul Singh,et al.  Proteus: a topology malleable data center network , 2010, Hotnets-IX.

[25]  Amin Vahdat,et al.  PortLand: a scalable fault-tolerant layer 2 data center network fabric , 2009, SIGCOMM '09.