Incentive Mechanisms for Resource Scaling-out Game of Stream Big Data Analytics

For stream big data analytics, a participated task always needs to scale out resources when its input data increases steeply. Typically, the resource scaling-out can be achieved by increasing the parallelism degree of the platform based on the experience. However, the resource scaling-out of each task produces additional cost not only from itself but also from other competitive tasks, which brings about great challenges to ensure the efficient utilization of resources. To solve it systematically, we consider the resource scaling-out as a non-cooperative game and formulate a total cost model including a risk function and a task execution time function. The total cost of resource scaling-out reflects the influence of topology structure for the benefit of a participated task. Then we introduce the concept of price of anarchy (POA) to this game and get its upper bounds under specific conditions to describe the efficiency loss of Nash equilibrium. Hence, two economic classic tax-based incentive policies: Pivotal Mechanism and Externality Mechanism are applied, to stimulate the participation of tasks. We make simulations in different scenarios including node degree and different characteristics of tasks. The simulations results show the influence of the topological structure and interdependent relationships of tasks for resource scaling-out game in the proposed scenarios and that the incentive mechanisms can effectively improve the performance of resource scaling-out.