Research on Job Scheduling Algorithm in Hadoop

On the basis of researching Fair Scheduling Strategy deeply in Hadoop cluster ，the Node Health Degree is defined by constructing the relationship function between node load and job fail rate, and a job scheduling algorithm based on Node Health Degree is proposed in this paper. Nodes are grouped, according to Node Health Degree, into three categories in order to assign corresponding job in accordance with load and guarantee resource load balance. By comparing with FIFO and Fair scheduling algorithm, the simulation results show that this algorithm can ensure to reduce job fail rate and improve cluster throughput.