ICT Innovations 2020. Machine Learning and Applications: 12th International Conference, ICT Innovations 2020, Skopje, North Macedonia, September 24–26, 2020, Proceedings

In this talk, we will present a few optimization principles that have been shown to be useful to address large-scale problems in machine learning. We will focus on recent variants of the stochastic gradient descent method that benefit from several acceleration mechanisms such as variance reduction and Nesterov’s extrapolation. We will discuss both theoretical results in terms of complexity analysis, and practical deployment of these approaches, demonstrating that even though Nesterov’s acceleration method is almost 40 years old, it is still highly relevant today.