Rewards, costs and flexibility in dynamic resource allocation: a stochastic optimal control approach

Various canonical forms of general resource allocation problemsarise naturally across a broad spectrum of computer systems andcommunication networks. As the complexities of these systemsand networks continue to grow, together with ubiquitous advancesin technology, new approaches and methods are required to effec-tively and efficiently solve these problems. Such environments of-ten consist of different types of resources that are allocated in com-bination to serve demand whose behavior over time is characterizedby different types of uncertainty and variability. Each type of re-source has a different reward and cost structure that ranges from thebest of a set of primary resource allocation options, having the high-est reward, highest cost and highest net-benefit, to a secondary re-source allocation option, having the lowest reward, lowest cost andlowest net-benefit. Each type of resource also has different struc-tures for the flexibility and cost of making changes to the allocationcapacity. The resource management optimization problem we con-sider consists of adaptively determining the primary and secondaryresource allocation capacities that serve the uncertain demand andthat maximize the expected net-benefit over a time horizon of in-terest based on the foregoing reward, cost and flexibility structuralproperties of the different types of resources.The general class of resource allocation problems studied in thispaper arises in a wide variety of application domains such as cloudcomputing and data center environments, computer and communi-cation networks, and energy-aware and smart power grid environ-ments, among many others. Across these and many other domain-specific resource allocation problems, there is a common need forthe dynamic adjustment of allocations among multiple types of re-sources, each with different structural properties, to satisfy time-varying and uncertain demand. Taking a financial mathematicsapproach that hedges against future risks associated with resourceallocation decisions and uncertain demand, we consider the under-lying fundamental stochastic optimal control problem where thedynamic control policy that allocates primary resource capacitiesto serve uncertain demand is a variational stochastic process withconditions on its derivative, which in turn determines the secondaryresource allocation capacity. The objective is to maximize the ex-pected discounted net-benefit over time based on the structural prop-