Distributed compilation system for high-speed software build processes

The idle time of personal computers has increased steadily due to the generalization of computer usage and cloud computing. Clustering research aims at utilizing idle computer resources for processing a variable workload on a large number of computers. The workload is processed continually despite the volatile status of the individual computer resources. This paper proposes a distributed compilation system for improving the processing speed of CPU-intensive software compilations. This significantly reduces the compilation time of mass sources by using the idle resources. We expect gains of up to 65% compared to non-distributed compilation systems.

[1]  Roger Impey,et al.  HTC scientific computing in a distributed cloud environment , 2013, Science Cloud '13.

[2]  Jin-Soo Kim,et al.  A low-overhead networking mechanism for virtualized high-performance computing systems , 2010, The Journal of Supercomputing.

[3]  David P. Anderson,et al.  BOINC: a system for public-resource computing and storage , 2004, Fifth IEEE/ACM International Workshop on Grid Computing.

[4]  Gordon Bell,et al.  What's next in high-performance computing? , 2002, CACM.

[5]  Andrew A. Chien,et al.  Resource Management for Rapid Application Turnaround on Enterprise Desktop Grids , 2004, Proceedings of the ACM/IEEE SC2004 Conference.

[6]  C.B.Ries,et al.  Code Generation Approaches for an Automatic Transformation of the Unified Modeling Language to the Berkeley Open Infrastructure for Network Computing Framework , 2013, SOCO 2013.

[7]  Miron Livny,et al.  Scheduling Mixed Workloads in Multi-grids: The Grid Execution Hierarchy , 2006, 2006 15th IEEE International Conference on High Performance Distributed Computing.

[8]  Aman Madaan,et al.  Design and implementation of a high performance computing system using distributed compilation , 2013, 2013 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[9]  Yves Robert,et al.  Scheduling divisible workloads on heterogeneous platforms , 2003, Parallel Comput..

[10]  Douglas Thain,et al.  A compiler toolchain for distributed data intensive scientific workflows , 2012 .

[11]  A. Kivity,et al.  kvm : the Linux Virtual Machine Monitor , 2007 .

[12]  Douglas Thain,et al.  Building Reliable Clients and Services , 2004, The Grid 2, 2nd Edition.

[13]  Kihong Kim,et al.  Optimizing multidimensional index trees for main memory access , 2001, SIGMOD '01.

[14]  Miron Livny,et al.  Managing network resources in Condor , 2000, Proceedings the Ninth International Symposium on High-Performance Distributed Computing.

[15]  Ed Burnette,et al.  Hello, Android: Introducing Google's Mobile Development Platform , 2009 .

[16]  Douglas Thain,et al.  Distributed computing in practice: the Condor experience , 2005, Concurr. Pract. Exp..

[17]  Pavel Tvrdík,et al.  Different Approaches to Distributed Compilation , 2012, 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops & PhD Forum.

[18]  Igor Sfiligoi,et al.  Pilot job accounting and auditing in Open Science Grid , 2008, 2008 9th IEEE/ACM International Conference on Grid Computing.

[19]  Franck Cappello,et al.  Cost-benefit analysis of Cloud Computing versus desktop grids , 2009, 2009 IEEE International Symposium on Parallel & Distributed Processing.

[20]  Thomas Hérault,et al.  DAGuE: A Generic Distributed DAG Engine for High Performance Computing , 2011, 2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and Phd Forum.

[21]  Seon Wook Kim,et al.  DiSCo: Distributed Scalable Compilation Tool for Heavy Compilation Workload , 2013, IEICE Trans. Inf. Syst..

[22]  Ami Marowka,et al.  The GRID: Blueprint for a New Computing Infrastructure , 2000, Parallel Distributed Comput. Pract..

[23]  Miron Livny,et al.  Checkpoint and Migration of UNIX Processes in the Condor Distributed Processing System , 1997 .

[24]  David G. Wonnacott,et al.  Distributed Shared Memory and Compiler-Induced Scalable Locality for Scalable Cluster Performance , 2012, 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012).

[25]  Miron Livny,et al.  Stork: making data placement a first class citizen in the grid , 2004, 24th International Conference on Distributed Computing Systems, 2004. Proceedings..