We consider a multi-armed bandit setting that is inspired by real-world applications in e-commerce. In our setting, there are a few types of users, each with a spec ific response to the different arms. When a user enters the system, his type is unknown to the decision m aker. The decision maker can either treat each user separately ignoring the previously observe d us rs, or can attempt to take advantage of knowing that only few types exist and cluster the users accor ding to their response to the arms. We devise algorithms that combine the usual exploration-expl oitation tradeoff with clustering of users and demonstrate the value of clustering. In the process of devel oping algorithms for the clustered setting, we propose and analyze simple algorithms for the setup where a d ecision maker knows that a user belongs to one of few types, but does not know which one.
[1]
J. Langford,et al.
The Epoch-Greedy algorithm for contextual multi-armed bandits
,
2007,
NIPS 2007.
[2]
Aleksandrs Slivkins,et al.
Contextual Bandits with Similarity Information
,
2009,
COLT.
[3]
Vianney Perchet,et al.
The multi-armed bandit problem with covariates
,
2011,
ArXiv.
[4]
H. Robbins,et al.
Asymptotically efficient adaptive allocation rules
,
1985
.
[5]
J. Bather,et al.
Multi‐Armed Bandit Allocation Indices
,
1990
.
[6]
Philippe Rigollet,et al.
Nonparametric Bandits with Covariates
,
2010,
COLT.
[7]
Peter Auer,et al.
Finite-time Analysis of the Multiarmed Bandit Problem
,
2002,
Machine Learning.