In this paper, a process-aware security task- scheduling algorithm is proposed as IOAware. The algorithm evaluates the performance of hardware in computing nodes, and estimates properties of a task while it is being executed. In subsequent assignments, the algorithm assigns tasks based on the performance of the TaskTracker and the task properties. To verify the theoretical feasibility of the proposed IOAware scheduling algorithm, a schedule model is proposed to implement the method. After applying the scheduling module in a Hadoop cluster with multiple experiments, the results show that, compared with the performances of the FIFO, computing capacity-scheduling, and fair-scheduling algorithms in terms of four aspects: response time, localization ratio of the data, system throughput, and system resources, IOAware scheduling algorithm can reach up to the disk IO effect of the Shared computing nodes, effectively reduce the execution time of the tasks and improve the throughput of the cluster.
[1]
Lei Wang,et al.
Research on Job Scheduling Algorithm in Hadoop
,
2011
.
[2]
Anirban Dasgupta,et al.
On scheduling in map-reduce and flow-shops
,
2011,
SPAA '11.
[3]
Douglas G. Down,et al.
An adaptive scheduling algorithm for dynamic heterogeneous Hadoop systems
,
2011,
CASCON.
[4]
Ming Zhong,et al.
Replication degree customization for high availability
,
2008,
Eurosys '08.
[5]
David E. Culler,et al.
The ganglia distributed monitoring system: design, implementation, and experience
,
2004,
Parallel Comput..
[6]
Rob Pike,et al.
Interpreting the data: Parallel analysis with Sawzall
,
2005,
Sci. Program..