Automated application component placement in data centers using mathematical programming

In this article we address the application component placement (ACP) problem for a data center. The problem is defined as follows: for a given topology of a network consisting of switches, servers and storage devices with varying capabilities, and for a given specification of a component-based distributed application, decide which physical server should be assigned to each application component, such that the application's processing, communication and storage requirements are satisfied without creating bottlenecks in the infrastructure, and that scarce resources are used most efficiently. We explain how the ACP problem differs from traditional task assignment in distributed systems, or existing grid scheduling problems. We describe our approach of formalizing this problem using a mathematical optimization framework and further formulating it as a mixed integer program (MIP). We then present our ACP solver using GAMS and CPLEX to automate the decision-making process. The solver was numerically tested on a number of examples, ranging from a 125-server real data center to a set of hypothetical data centers with increasing size. In all cases the ACP solver found an optimal solution within a reasonably short time. In a numerical simulation comparing our solver to a random selection algorithm, our solver resulted in much more efficient use of scarce network resources and allowed more applications to be placed in the same infrastructure.

[1]  Xiaoyun Zhu,et al.  Optimal resource assignment in Internet data centers , 2001, MASCOTS 2001, Proceedings Ninth International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems.

[2]  Akhil Sahai,et al.  Automated policy-based resource construction in utility computing environments , 2004, 2004 IEEE/IFIP Network Operations and Management Symposium (IEEE Cat. No.04CH37507).

[3]  Ram Swaminathan,et al.  Ergastulum: Quickly fi nding near-optimal storage system designs , 2001 .

[4]  G. Nemhauser,et al.  Integer Programming , 2020 .

[5]  Dirk Beyer,et al.  Quartermaster - a resource utility system , 2005, 2005 9th IFIP/IEEE International Symposium on Integrated Network Management, 2005. IM 2005..

[6]  Wesley W. Chu,et al.  Task Allocation in Distributed Data Processing , 1980, Computer.

[7]  Eric Anderson,et al.  Quickly finding near-optimal storage designs , 2005, TOCS.

[8]  Robert Ricci,et al.  A solver for the network testbed mapping problem , 2003, CCRV.

[9]  Xiaoyun Zhu,et al.  A Mathematical Optimization Approach for Resource Allocation in Large Scale Data Centers , 2002 .

[10]  Claudio Bartolini,et al.  Market-Based Resource Allocation for Utility Data Centers , 2003 .

[11]  Lawrence W. Dowdy,et al.  Comparative Models of the File Assignment Problem , 1982, CSUR.

[12]  Shahid H. Bokhari,et al.  A Shortest Tree Algorithm for Optimal Assignments Across Space and Time in a Distributed Processor System , 1981, IEEE Transactions on Software Engineering.

[13]  Radia Perlman,et al.  Interconnections: Bridges, Routers, Switches, and Internetworking Protocols , 1999 .

[14]  Ian Foster,et al.  The Grid 2 - Blueprint for a New Computing Infrastructure, Second Edition , 1998, The Grid 2, 2nd Edition.

[15]  Subhash Suri,et al.  Bandwidth-Constrained Allocation in Grid Computing , 2003, Algorithmica.

[16]  Terence Kelly,et al.  Generalized knapsack solvers for multi-unit combinatorial auctions: analysis and application to computational resource allocation , 2004, AAMAS'04.

[17]  Tad Hogg,et al.  Spawn: A Distributed Computational Economy , 1992, IEEE Trans. Software Eng..

[18]  Ami Marowka,et al.  The GRID: Blueprint for a New Computing Infrastructure , 2000, Parallel Distributed Comput. Pract..

[19]  Julie Ward,et al.  Appia: Automatic Storage Area Network Fabric Design , 2002, FAST.

[20]  Virginia Mary Lo,et al.  Heuristic Algorithms for Task Assignment in Distributed Systems , 1988, IEEE Trans. Computers.

[21]  Biswanath Mukherjee,et al.  Heuristic algorithms for constructing optimized structures of linear multihop lightwave networks , 1994, IEEE Trans. Commun..

[22]  Elwood S. Buffa,et al.  A Heuristic Algorithm and Simulation Approach to Relative Location of Facilities , 1963 .

[23]  Harold S. Stone,et al.  Multiprocessor Scheduling with the Aid of Network Flow Algorithms , 1977, IEEE Transactions on Software Engineering.