Reconfigurable network testbed for evaluation of datacenter topologies

Software-defined networking combined with distributed and parallel applications has the potential to deliver optimized application performance at runtime. In order to investigate this enhancement and design future implementation, a datacenter with a programmable topology integrated with application state is needed. Towards this goal, we introduce the Flow Optimized Route Configuration Engine (FORCE). The FORCE is an emulated datacenter testbed with a programmable interconnection controlled by an SDN controller. We also utilize Hadoop as a case study of distributed and parallel applications along with a simulated Hadoop shuffle traffic generator. The testbed provides initial experimental evidence of support to our hypothesis for future SDN research. Our experiments on the testbed show a difference in application runtime a factor of over 2.5 times on shuffle traffic for Hadoop MapReduce jobs and the potential for significant speedup in warehouse scale data centers.

[1]  Anees Shaikh,et al.  EPIC: Platform-as-a-Service Model for Cloud Networking , 2011 .

[2]  Haitao Wu,et al.  BCube: a high performance, server-centric network architecture for modular data centers , 2009, SIGCOMM '09.

[3]  David A. Maltz,et al.  Network traffic characteristics of data centers in the wild , 2010, IMC '10.

[4]  Sujata Banerjee,et al.  ElasticTree: Saving Energy in Data Center Networks , 2010, NSDI.

[5]  Jeffrey C. Mogul,et al.  SPAIN: COTS Data-Center Ethernet for Multipathing over Arbitrary Topologies , 2010, NSDI.

[6]  Robert B. Ross,et al.  PVFS: A Parallel File System for Linux Clusters , 2000, Annual Linux Showcase & Conference.

[7]  Minyi Guo,et al.  OFScheduler: A Dynamic Network Optimizer for MapReduce in Heterogeneous Cluster , 2013, International Journal of Parallel Programming.

[8]  Richard Wang,et al.  OpenFlow-Based Server Load Balancing Gone Wild , 2011, Hot-ICE.

[9]  Stuart Bailey,et al.  Hadoop Acceleration in an OpenFlow-Based Cluster , 2012, 2012 SC Companion: High Performance Computing, Networking Storage and Analysis.

[10]  Hai Jin,et al.  LEEN: Locality/Fairness-Aware Key Partitioning for MapReduce in the Cloud , 2010, 2010 IEEE Second International Conference on Cloud Computing Technology and Science.

[11]  Alex C. Snoeren,et al.  Topology Switching for Data Center Networks , 2011, Hot-ICE.

[12]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[13]  Ming Zhang,et al.  Understanding data center traffic characteristics , 2010, CCRV.

[14]  Wolfgang Kellerer,et al.  NetServ: Active Networking 2.0 , 2011, 2011 IEEE International Conference on Communications Workshops (ICC).

[15]  Mohammad Hammoud,et al.  Locality-Aware Reduce Task Scheduling for MapReduce , 2011, 2011 IEEE Third International Conference on Cloud Computing Technology and Science.

[16]  Weikuan Yu,et al.  Hadoop acceleration through network levitated merge , 2011, 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[17]  Amin Vahdat,et al.  Hedera: Dynamic Flow Scheduling for Data Center Networks , 2010, NSDI.

[18]  Mohammad Hammoud,et al.  Center-of-Gravity Reduce Task Scheduling to Lower MapReduce Network Traffic , 2012, 2012 IEEE Fifth International Conference on Cloud Computing.

[19]  Nick McKeown,et al.  OpenFlow: enabling innovation in campus networks , 2008, CCRV.

[20]  Nick McKeown,et al.  Leveraging SDN layering to systematically troubleshoot networks , 2013, HotSDN '13.

[21]  Anees Shaikh,et al.  Programming your network at run-time for big data applications , 2012, HotSDN '12.

[22]  D. Panda,et al.  Can High-Performance Interconnects Benefit Hadoop Distributed File System ? , 2010 .

[23]  Amin Vahdat,et al.  A scalable, commodity data center network architecture , 2008, SIGCOMM '08.

[24]  Renato J. O. Figueiredo,et al.  A case for grid computing on virtual machines , 2003, 23rd International Conference on Distributed Computing Systems, 2003. Proceedings..