Minimal Coflow Routing and Scheduling in OpenFlow-Based Cloud Storage Area Networks

Researches affirm that coflow scheduling/routing substantially shortens the average application inner communication time in data center networks(DCNs). The commonly desirable critical features of existing coflow scheduling/routing framework includes (1) coflow scheduling, (2) coflow routing, and (3) per-flow rate-limiting. However, to provide the 3 features, existing frameworks require customized computing frameworks, customized operating systems, or specific external commercial monitoring frameworks on software-defined networking(SDN) switches. These requirements defer or even prohibit the deployment of coflow scheduling/routing in production DCNs. In this paper, we design a coflow scheduling and routing framework, MinCOF which has minimal requirements on hosts and switches for cloud storage area networks(SANs) based on OpenFlow SDN. MinCOF accommodates all critical features of coflow scheduling/routing from previous works. The deployability in production environment is especially taken into consideration. The OpenFlow architecture is capable of processing the traffic load in a cloud SAN. Not necessary requirements for hosts from existing frameworks are migrated to the mature commodity OpenFlow 1.3 Switch and our coflow scheduler. Transfer applications on hosts only need slight enhancements on their existing connection establishment and progress reporting functions. Evaluations reveal that MinCOF decreases the average coflow completion time (CCT) by 12.94% compared to the latest OpenFlow-based coflow scheduling and routing framework.

[1]  Ion Stoica,et al.  Efficient coflow scheduling with Varys , 2015, SIGCOMM.

[2]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[3]  Matthew Mathis,et al.  The macroscopic behavior of the TCP congestion avoidance algorithm , 1997, CCRV.

[4]  Robert L. Grossman,et al.  The Design of a Community Science Cloud: The Open Science Data Cloud Perspective , 2012, 2012 SC Companion: High Performance Computing, Networking Storage and Analysis.

[5]  Nathan Lewis,et al.  BIC-LSU: Big Data Research Integration with Cyberinfrastructure for LSU , 2016, XSEDE.

[6]  Bo Li,et al.  Tailor: Trimming Coflow Completion Times in Datacenter Networks , 2016, 2016 25th International Conference on Computer Communication and Networks (ICCCN).

[7]  Suman Kumar,et al.  FaLL: A fair and low latency queuing scheme for data center networks , 2015, 2015 International Conference on Computing, Networking and Communications (ICNC).

[8]  Michael I. Jordan,et al.  Managing data transfers in computer clusters with orchestra , 2011, SIGCOMM.

[9]  Amin Vahdat,et al.  A scalable, commodity data center network architecture , 2008, SIGCOMM '08.

[10]  Ion Stoica,et al.  Coflow: a networking abstraction for cluster applications , 2012, HotNets-XI.

[11]  Mahidhar Tatineni,et al.  Storage utilization in the long tail of science , 2015, XSEDE.

[12]  Sheng Wang,et al.  Rapier: Integrating routing and scheduling for coflow-aware data center networks , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).

[13]  Aart J. C. Bik,et al.  Pregel: a system for large-scale graph processing , 2010, SIGMOD Conference.