Optimal online algorithms for MapReduce scheduling on two uniform machines

We study online scheduling on two uniform machines in the MapReduce system. Each job consists of two sets of tasks, namely the map tasks and reduce tasks. A job’s reduce tasks can only be processed after all its map tasks are finished. The map tasks are fractional, i.e., they can be arbitrarily split and processed on different machines in parallel. Our goal is to find a schedule that minimizes the makespan. We consider two variants of the problem, namely the cases involving preemptive reduce tasks and non-preemptive reduce tasks. We provide lower bounds for both variants. For preemptive reduce tasks, we present an optimal online algorithm with a competitive ratio of $$\frac{\sqrt{s^{2}+2s+5}+1-s}{2}$$s2+2s+5+1-s2, where $$s\ge 1$$s≥1 is the ratio between the speeds of the two machines. For non-preemptive reduce tasks, we show that the $${ LS}$$LS-like algorithm is optimal and its competitive ratio is $$\frac{2s+1}{s+1}$$2s+1s+1 if $$s<\frac{1+\sqrt{5}}{2}$$s<1+52 and $$\frac{s+1}{s}$$s+1s if $$s\ge \frac{1+\sqrt{5}}{2}$$s≥1+52.

[1]  Deying Li,et al.  Makespan minimization for MapReduce systems with different servers , 2017, Future Gener. Comput. Syst..

[2]  Wei Zhou,et al.  An Optimal Preemptive Algorithm for Online MapReduce Scheduling on Two Parallel Machines , 2018, Asia Pac. J. Oper. Res..

[3]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[4]  Yin-Feng Xu,et al.  Online makespan minimization in MapReduce-like systems with complex reduce tasks , 2017, Optim. Lett..

[5]  Donglei Du,et al.  Preemptive on-line scheduling for two uniform processors , 1998, Oper. Res. Lett..

[6]  Funda Ergün,et al.  Online load balancing for MapReduce with skewed data input , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[7]  Ness B. Shroff,et al.  A new analytical technique for designing provably efficient MapReduce schedulers , 2013, 2013 Proceedings IEEE INFOCOM.

[8]  Gerhard J. Woeginger,et al.  Randomized online scheduling on two uniform machines , 2001, SODA '99.

[9]  Murali S. Kodialam,et al.  Scheduling in mapreduce-like systems for fast completion time , 2011, 2011 Proceedings IEEE INFOCOM.

[10]  Anirban Dasgupta,et al.  On scheduling in map-reduce and flow-shops , 2011, SPAA '11.

[11]  Yin-Feng Xu,et al.  Online MapReduce processing on two identical parallel machines , 2018, J. Comb. Optim..

[12]  Yin-Feng Xu,et al.  Online MapReduce scheduling problem of minimizing the makespan , 2017, J. Comb. Optim..

[13]  Deying Li,et al.  Minimizing makespan and total completion time in MapReduce-like systems , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.