On the Rate of Convergence of Regularized Learning in Games: From Bandits and Uncertainty to Optimism and Beyond