A stochastic bandit algorithm for scratch games

Stochastic multi-armed bandit algorithms are used to solve the exploration and exploitation dilemma in sequential optimization problems. The algorithms based on upper condence bounds oer strong theoretical guarantees, they are easy to implement and ecient in practice. We considers a new bandit setting, called \scratch-games", where arm budgets are limited and reward are drawn without replacement. Using Sering inequality, we propose an upper condence bound algorithm adapted to this setting. We show that the bound of expectation to play a suboptimal arm is lower than the one of UCB1 policy. We illustrate this result on both synthetic problems and realistic problems (ad-serving and emailing campaigns optimization).