Implicit Adaptation to Low Rank Structure in Online Learning

This paper is about the relationship between regret (in online learning) and the rank of an ensemble’s loss matrix Y. Recently, several new algorithms have been developed to exploit low rank structure in Y. Unfortunately, each of these is not known to be order minimax optimal outside of specialized settings. This paper explores through simulation whether this apparent difficulty in achieving minimax optimality is because highly specialized algorithms are required. We observe that a horizon-adaptive hedge algorithm appears to exploit low rank structure effectively, suggesting that algorithms do not have to explicitly work to exploit low rank structure.