Forward-backward-forward methods with variance reduction for stochastic variational inequalities

We develop a new stochastic algorithm with variance reduction for solving pseudo-monotone stochastic variational inequalities. Our method builds on Tseng's forward-backward-forward (FBF) algorithm, which is known in the deterministic literature to be a valuable alternative to Korpelevich's extragradient method when solving variational inequalities over a convex and closed set governed by pseudo-monotone, Lipschitz continuous operators. The main computational advantage of Tseng's algorithm is that it relies only on a single projection step and two independent queries of a stochastic oracle. Our algorithm incorporates a variance reduction mechanism and leads to almost sure (a.s.) convergence to an optimal solution. To the best of our knowledge, this is the first stochastic look-ahead algorithm achieving this by using only a single projection at each iteration.

[1]  A. Juditsky,et al.  Solving variational inequalities with Stochastic Mirror-Prox algorithm , 2008, 0809.0815.

[2]  Uday V. Shanbhag,et al.  On the Characterization of Solution Sets of Smooth and Nonsmooth Convex Stochastic Nash Games , 2011, SIAM J. Optim..

[3]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[4]  G. M. Korpelevich.  The extragradient method for finding saddle points and other problems , 1976 .

[5]  Gersende Fort,et al.  On Perturbed Proximal Gradient Algorithms , 2014, J. Mach. Learn. Res..

[6]  Francisco Facchinei,et al.  Convex Optimization, Game Theory, and Variational Inequality Theory , 2010, IEEE Signal Processing Magazine.

[7]  O. Nelles,et al.  An Introduction to Optimization , 1996, IEEE Antennas and Propagation Magazine.

[8]  Houyuan Jiang,et al.  Stochastic Approximation Approaches to the Stochastic Variational Inequality Problem , 2008, IEEE Transactions on Automatic Control.

[9]  Alfredo N. Iusem,et al.  Extragradient Method with Variance Reduction for Stochastic Variational Inequalities , 2017, SIAM J. Optim..

[10]  R. Tyrrell Rockafellar,et al.  Asymptotic Theory for Solutions in Statistical Estimation and Stochastic Programming , 1993, Math. Oper. Res..

[11]  Alejandro Jofré,et al.  On variance reduction for stochastic smooth convex optimization with multiplicative noise , 2017, Math. Program..

[12]  Lorenzo Rosasco,et al.  Stochastic Forward–Backward Splitting for Monotone Inclusions , 2016, J. Optim. Theory Appl..

[13]  N. Bingham.  Probability Theory: An Analytic View , 2002 .

[14]  Guanghui Lan,et al.  On the convergence properties of non-Euclidean extragradient methods for variational inequalities with generalized monotone operators , 2013, Comput. Optim. Appl..

[15]  Mathias Staudigl,et al.  Learning in time-varying games , 2018, ArXiv.

[16]  Yunmei Chen,et al.  Stochastic Accelerated Alternating Direction Method of Multipliers with Importance Sampling , 2018, Journal of Optimization Theory and Applications.

[17]  Carlos S. Kubrusly,et al.  Stochastic approximation algorithms and applications , 1973, CDC 1973.

[18]  Wei Yu,et al.  Fractional Programming for Communication Systems—Part I: Power Control and Beamforming , 2018, IEEE Transactions on Signal Processing.

[19]  M. Solodov,et al.  A New Projection Method for Variational Inequality Problems , 1999 .

[20]  B. Stengel,et al.  COMPUTING EQUILIBRIA FOR TWO-PERSON GAMES , 1996 .

[21]  Paul Tseng,et al.  A Modified Forward-backward Splitting Method for Maximal Monotone Mappings 1 , 1998 .

[22]  Patrick L. Combettes,et al.  Stochastic Quasi-Fejér Block-Coordinate Fixed Point Iterations with Random Sweeping , 2014 .

[23]  Emre Telatar,et al.  Capacity of Multi-antenna Gaussian Channels , 1999, Eur. Trans. Telecommun..

[24]  Angelia Nedic,et al.  On smoothing, regularization, and averaging in stochastic approximation methods for stochastic variational inequality problems , 2017, Math. Program..

[25]  Andrea J. Goldsmith,et al.  Energy-efficiency of MIMO and cooperative MIMO techniques in sensor networks , 2004, IEEE Journal on Selected Areas in Communications.

[26]  Panayotis Mertikopoulos,et al.  Learning to Be Green: Robust Energy Efficiency Maximization in Dynamic MIMO–OFDM Systems , 2015, IEEE Journal on Selected Areas in Communications.

[27]  Jorge Nocedal,et al.  Optimization Methods for Large-Scale Machine Learning , 2016, SIAM Rev..

[28]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[29]  Geoffrey Ye Li,et al.  A survey of energy-efficient wireless communications , 2013, IEEE Communications Surveys & Tutorials.

[30]  Mathias Staudigl,et al.  Stochastic Mirror Descent Dynamics and Their Convergence in Monotone Variational Inequalities , 2017, Journal of Optimization Theory and Applications.

[31]  M. Hata,et al.  Empirical formula for propagation loss in land mobile radio services , 1980, IEEE Transactions on Vehicular Technology.

[32]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[33]  Alexander Shapiro,et al.  Lectures on Stochastic Programming: Modeling and Theory , 2009 .

[34]  Gerhard Fettweis,et al.  Framework for Link-Level Energy Efficiency Optimization with Informed Transmitter , 2011, IEEE Transactions on Wireless Communications.

[35]  Uday V. Shanbhag,et al.  Distributed Computation of Equilibria in Monotone Nash Games via Iterative Regularization Techniques , 2012, SIAM J. Optim..

[36]  Zhengyuan Zhou,et al.  Learning in games with continuous action sets and unknown payoff functions , 2019, Math. Program..