SARN: A scalable resource managing framework for YARN

With the fast growth of Internet, we have entered the era of big data. In the big data era, Hadoop Yet Another Resource Negotiator (YARN) is one of the common used framework for big data processing. YARN provides explicit support for programming model diversity, so multiple frameworks such as Storm, Hbase and Hive can run as applications on YARN [1]. In order to make better use of hardware resources and improve the cluster efficiency, a scalable resource managing framework (SARN) is presented. SARN includes the dynamic container and a quick deploy component for elastically expanding or shrinking the number of YARN'S work node to meet the actual resource needs of multiple tasks. I do experiment under both idle mode (the cluster is idle) and eager mode (the cluster resource is relatively not enough to meet the application's requirements). The experimental result shows that the approach can improve the performance applications under both situations.