VirtualKnotter: Online Virtual Machine Shuffling for Congestion Resolving in Virtualized Datacenter

Our measurements on production data center traffic together with recently reported results suggest that data center networks suffer from long-lived congestion caused by core network over subscription and unbalanced workload placement. In contrast to traditional traffic engineering approaches that optimize flow routing, in this paper, we explore the opportunity to address the continuous congestion via optimizing VM placement in virtualized data centers. To this end, we present Virtual Knotter, an efficient online VM placement algorithm to reduce congestion with controllable VM migration traffic as well as low time complexity. Our evaluation with both real and synthetic traffic patterns shows that Virtual Knotter performs close to the baseline algorithm in terms of link unitization, with only 5%-10% migration traffic of the baseline algorithm. Furthermore, Virtual Knotter decreases link congestion time by 53% for the production data center traffic.

[1]  Amin Vahdat,et al.  Hedera: Dynamic Flow Scheduling for Data Center Networks , 2010, NSDI.

[2]  Albert G. Greenberg,et al.  COPE: traffic engineering in dynamic networks , 2006, SIGCOMM.

[3]  Haitao Wu,et al.  BCube: a high performance, server-centric network architecture for modular data centers , 2009, SIGCOMM '09.

[4]  Craig MacDonald,et al.  Comparing Distributed Indexing: To MapReduce or Not? , 2009, LSDS-IR@SIGIR.

[5]  Ramana Rao Kompella,et al.  On the efficacy of fine-grained traffic splitting protocolsin data center networks , 2011, SIGCOMM 2011.

[6]  Christian Engelmann,et al.  Proactive fault tolerance for HPC with Xen virtualization , 2007, ICS '07.

[7]  Luís Henrique Maciel Kosmalski Costa,et al.  Online traffic-aware virtual machine placement in data center networks , 2012, 2012 Global Information Infrastructure and Networking Symposium (GIIS).

[8]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[9]  Ming Zhang,et al.  The Case for Fine-Grained Traffic Engineering in Data Centers , 2010, INM/WREN.

[10]  Brian W. Kernighan,et al.  An efficient heuristic procedure for partitioning graphs , 1970, Bell Syst. Tech. J..

[11]  Hai Jin,et al.  Live virtual machine migration with adaptive, memory compression , 2009, 2009 IEEE International Conference on Cluster Computing and Workshops.

[12]  GhemawatSanjay,et al.  The Google file system , 2003 .

[13]  Paramvir Bahl,et al.  Augmenting data center networks with multi-gigabit wireless links , 2011, SIGCOMM.

[14]  Michael I. Jordan,et al.  Managing data transfers in computer clusters with orchestra , 2011, SIGCOMM.

[15]  Kartik Gopalan,et al.  Explicit coordination to prevent congestion in data center networks , 2011, Cluster Computing.

[16]  Ramana Rao Kompella,et al.  On the efficacy of fine-grained traffic splitting protocols in data center networks , 2012, SIGMETRICS.

[17]  Karsten Schwan,et al.  VirtualPower: coordinated power management in virtualized enterprise systems , 2007, SOSP.

[18]  Scott Shenker,et al.  Spark: Cluster Computing with Working Sets , 2010, HotCloud.

[19]  Fabrizio Silvestri,et al.  Workshop on large-scale distributed systems for information retrieval , 2007, SIGF.

[20]  Joachim Hammer,et al.  A new hierarchical clustering model for speeding up the reconciliation of xml-based, semistructured data in mediation systems , 2001 .

[21]  Yuan Yu,et al.  Dryad: distributed data-parallel programs from sequential building blocks , 2007, EuroSys '07.

[22]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[23]  Jason Cong,et al.  Multi-way partitioning using bi-partition heuristics , 2000, ASP-DAC '00.

[24]  Kang-Won Lee,et al.  Application-aware virtual machine migration in data centers , 2011, 2011 Proceedings IEEE INFOCOM.

[25]  Liang Zhong,et al.  EnaCloud: An Energy-Saving Application Live Placement Approach for Cloud Computing Environments , 2009, 2009 IEEE International Conference on Cloud Computing.

[26]  Rainer E. Burkard,et al.  Selected topics on assignment problems , 2002, Discret. Appl. Math..

[27]  Fung Po Tso,et al.  Implementing Scalable, Network-Aware Virtual Machine Migration for Cloud Data Centers , 2013, 2013 IEEE Sixth International Conference on Cloud Computing.

[28]  Albert G. Greenberg,et al.  The nature of data center traffic: measurements & analysis , 2009, IMC '09.

[29]  Vijay Mann,et al.  Remedy: Network-Aware Steady State VM Management for Data Centers , 2012, Networking.

[30]  Lei Shi,et al.  Dcell: a scalable and fault-tolerant network structure for data centers , 2008, SIGCOMM '08.

[31]  David A. Maltz,et al.  Network traffic characteristics of data centers in the wild , 2010, IMC '10.

[32]  Edith Cohen,et al.  Making intra-domain routing robust to changing and uncertain traffic demands: understanding fundamental tradeoffs , 2003, SIGCOMM '03.

[33]  Michiel H. M. Smid,et al.  A linear-space algorithm for distance preserving graph embedding , 2009, Comput. Geom..

[34]  Andrew Warfield,et al.  Live migration of virtual machines , 2005, NSDI.

[35]  Giuseppe Di Battista,et al.  26 Computer Networks , 2004 .

[36]  Vasileios Pappas,et al.  Improving the Scalability of Data Center Networks with Traffic-aware Virtual Machine Placement , 2010, 2010 Proceedings IEEE INFOCOM.

[37]  Albert G. Greenberg,et al.  VL2: a scalable and flexible data center network , 2009, SIGCOMM '09.

[38]  C. P. Ravikumar,et al.  Parallel Methods for Vlsi Layout Design , 1998, IEEE Concurrency.

[39]  Amin Vahdat,et al.  PortLand: a scalable fault-tolerant layer 2 data center network fabric , 2009, SIGCOMM '09.

[40]  Angela L. Chiu,et al.  Overview and Principles of Internet Traffic Engineering , 2002, RFC.

[41]  Andrzej Kochut,et al.  Dynamic Placement of Virtual Machines for Managing SLA Violations , 2007, 2007 10th IFIP/IEEE International Symposium on Integrated Network Management.