DScheduler: Dynamic Network Scheduling Method for MapReduce in Distributed Controllers

MapReduce is the most widely used distributed computing framework due to its excellent parallelism and scalability in dealing with large-scale data. It is one of the most important research point in distributed computing field to improve the performance of MapReduce application in datacenter network. OpenFlow protocol makes it possible to schedule network resource dynamically to provide better link bandwidth for shuffle traffic. Current OpenFlow-based scheduling method runs on a single controller, which cannot meet the needs of excessive switch requests in large scale data center networks. The performance of those scheduling method will decrease obviously due to some conflict problem when they run on distributed controllers. This paper proposed DScheduler, a dynamic network scheduling method for distributed controllers. DScheduler is running as an application on each SDN controller and avoid a majority of conflict problems in scheduling with small cost by using lock and communication between each controller. We implement a prototype system on Floodlight to demonstrate our design and test the performance. Experimental results show that DScheduler has a significant effect on decreasing the occurrence times of conflict situations and improving the performance of openflow-based scheduling method on distributed controllers.