Scheduling Algorithms for Distributed Cosmic Ray Detection Using Apache Mesos

This article presents two scheduling algorithms applied to the processing of astronomical images to detect cosmic rays on distributed memory high performance computing systems. We extend our previous article that proposed a parallel approach to improve processing times on image analysis using the Image Reduction and Analysis Facility IRAF software and the Docker project over Apache Mesos. By default, Mesos introduces a simple list scheduling algorithm where the first available task is assigned to the first available processor. On this paper we propose two alternatives for reordering the tasks allocation in order to improve the computational efficiency. The main results show that it is possible to reduce the makespan getting a speedup = 4.31 by adjusting how jobs are assigned and using Uniform processors.

[1]  K. Mani Chandy,et al.  A comparison of list schedules for parallel processing systems , 1974, Commun. ACM.

[2]  Edward G. Coffman,et al.  Algorithms minimizing mean flow time: schedule-length properties , 1976, Acta Informatica.

[3]  Francisco Brasileiro,et al.  Grid Computing for Bag of Tasks Applications , 2003 .

[4]  Navtej Singh,et al.  Parallel Astronomical Data Processing with Python: Recipes for multicore machines , 2013, Astron. Comput..

[5]  Annamária Kovács,et al.  Tighter approximation bounds for LPT scheduling in two special cases , 2006, J. Discrete Algorithms.

[6]  Ellis Horowitz,et al.  Exact and Approximate Algorithms for Scheduling Nonidentical Processors , 1976, JACM.

[7]  Milton Halem,et al.  Cloud Computing for Satellite Data Processing on High End Compute Clusters , 2009, 2009 IEEE International Conference on Cloud Computing.

[8]  Sergio Nesmachnow,et al.  Parallel multiobjective evolutionary algorithms for batch scheduling in heterogeneous computing and grid systems , 2013, Computational Optimization and Applications.

[9]  Mehraj Ali,et al.  Implementation of image processing system using handover technique with map reduce based on big data in the cloud environment , 2016, Int. Arab J. Inf. Technol..

[10]  Joseph Y.-T. Leung,et al.  Handbook of Scheduling: Algorithms, Models, and Performance Analysis , 2004 .

[11]  E.L. Lawler,et al.  Optimization and Approximation in Deterministic Sequencing and Scheduling: a Survey , 1977 .

[12]  Magdalena Balazinska,et al.  Astronomy in the Cloud: Using MapReduce for Image Co-Addition , 2010, ArXiv.

[13]  Ronald L. Graham,et al.  Bounds on Multiprocessing Timing Anomalies , 1969, SIAM Journal of Applied Mathematics.

[14]  E. O. Oyetunji,et al.  Some Common Performance Measures in Scheduling Problems: Review Article , 2009 .