Latent-Variable MDP Models for Adapting the Interaction Environment of Diverse Users

Interactive interfaces are a common feature of many systems ranging from field robotics to video games. In many applications, these interfaces are targeted at a diverse and heterogeneous set of potential users and correspondingly there is substantial variety in what types of interfaces can be made available (e.g., KEYBOARD vs. joysticks) and how they may be configured (e.g., sensitivity levels). In this paper we are interested in solving the problem of personalizing an interface such that it adapts to present the user with a variation that is optimal with respect to their traits, such as skill level. We pose this problem by modelling the user as a parametrized Markov Decision Process (MDP), wherein the transition dynamics within a task depend on the latent skill level of the user. This notion of adapting at the level of action sets, picking optimally from a potentially very large space of action sets, is novel and provides a natural solution to the interface personalization problem. Our solution to this problem involves a latent variable formulation wherein there are hidden personality traits, such as a user skill-level, which are implicitly associated with a specific (optimal) choice of action set. We present an algorithm that iteratively eliminates potential action sets to quickly arrive at the action set optimally suited to a particular user. We evaluate this procedure in an experiment involving a simulated remote navigation domain, demonstrating that the combination of latent variable user models and type elimination outperforms baselines that do not model the diversity of user actions in this way.