Combinatorial Sleeping Bandits With Fairness Constraints

The multi-armed bandit (MAB) model has been widely adopted for studying many practical optimization problems (network resource allocation, ad placement, crowdsourcing, etc.) with unknown parameters. The goal of the player (i.e., the decision maker) here is to maximize the cumulative reward in the face of uncertainty. However, the basic MAB model neglects several important factors of the system in many real-world applications, where multiple arms (i.e., actions) can be simultaneously played and an arm could sometimes be “sleeping” (i.e., unavailable). Besides reward maximization, ensuring <italic>fairness</italic> is also a key design concern in practice. To that end, we propose a new <italic>Combinatorial Sleeping MAB model with Fairness constraints</italic>, called <italic>CSMAB-F</italic>, aiming to address the aforementioned crucial modeling issues. The objective is now to maximize the reward while satisfying the fairness requirement of a minimum selection fraction for each individual arm. To tackle this new problem, we extend an online learning algorithm, called <italic>Upper Confidence Bound (UCB)</italic>, to deal with a critical tradeoff between <italic>exploitation</italic> and <italic>exploration</italic> and employ the virtual queue technique to properly handle the fairness constraints. By carefully integrating these two techniques, we develop a new algorithm, called <italic>Learning with Fairness Guarantee (LFG)</italic>, for the CSMAB-F problem. Further, we rigorously prove that not only LFG is <italic>feasibility-optimal</italic>, but it also has a time-average <italic>regret</italic> upper bounded by <inline-formula><tex-math notation="LaTeX">$\frac{N}{2 \eta } + \frac{\beta _1 \sqrt{m N T \log {T}}+ \beta _2~N}{T}$</tex-math></inline-formula>, where <inline-formula><tex-math notation="LaTeX">$N$</tex-math></inline-formula> is the total number of arms, <inline-formula><tex-math notation="LaTeX">$m$</tex-math></inline-formula> is the maximum number of arms that can be simultaneously played, <inline-formula><tex-math notation="LaTeX">$T$</tex-math></inline-formula> is the time horizon, <inline-formula><tex-math notation="LaTeX">$\beta _1$</tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX">$\beta _2$</tex-math></inline-formula> are constants, and <inline-formula><tex-math notation="LaTeX">$\eta$</tex-math></inline-formula> is a design parameter that we can tune. Finally, we perform extensive simulations to corroborate the effectiveness of the proposed algorithm. Interestingly, the simulation results reveal an important tradeoff between the regret and the speed of convergence to a point satisfying the fairness constraints.