Multi-Armed Bandits With Combinatorial Strategies Under Stochastic Bandits

We consider the following linearly combinatorial multiarmed bandits (MABs) problem. In a discrete time system, the re are K unknown random variables (RVs), i.e., arms, each evolving as an i.i.d stochastic process over time. At each time slot, w e select a set ofN (N ≤ K) RVs, i.e., strategy, subject to an arbitrarily constraint. We then gain a reward that is a linear combination of observations on selected RVs. Our goal is to minimize the regret, defined as the difference between the summed reward obtained by an optimal static policy that knew the mean of eac h RV, and that obtained by a specified learning policy that does not know. A prior result for this problem has achieved zero regret (the expect of regret over time approaches zero when t ime goes to infinity), but dependent on probability distribution of strategies generated by the learning policy. The regret bec omes arbitrarily large if the difference between the reward of the best and second best strategy approaches zero. Meanwhile, when t here are exponential number of combinations, naive extension of a prior distribution-free policy would cause poor performance in terms of regret, computation and space complexity. We propo se an efficient Distribution-Free Learning (DFL) policy that a chieves zero regret without dependence on probability distribution of strategies. Our learning policy only requires time and spac e complexity O(K). When the linear combination is involved with NP-hard problems, our policy provides a flexible scheme to choose possible approximation algorithms to solve the prob lem efficiently while retaining zero regret.