Randomized learning: Generalization performance of old and new theoretically grounded algorithms

Abstract In the context of assessing the generalization abilities of a randomized model or learning algorithm, PAC-Bayes and Differential Privacy (DP) theories are the state-of-the-art tools. For this reason, in this paper, we will develop tight DP-based generalization bounds, which improve over the current state-of-the-art ones both in terms of constants and rate of convergence. Moreover, we will also prove that some old and new randomized algorithm, show better generalization performances with respect to their non private counterpart, if the DP is exploited for assessing their generalization ability. Results on a series of algorithms and real world problems show the practical validity of the achieved theoretical results.