Forward-backward splitting techniques: theory and applications

This dissertation expands the ordinary forward-backward splitting method for finding a zero point of a maximal monotone operator to very generalized FBS methods and also develops the convergence theory for both ordinary and extended methods. In addition, some algorithm and problem structure analyses and application to large-scale convex optimalization (extended linear-quadratic programming, optimal control, stochastic programming, etc.), MPC controller design in chemical engineering, variational inequalities, and so on, are covered. Many related theoretical results are presented. In this dissertation, we mainly consider solving large-scale convex saddle point problems which involve dynamic systems, or stochastic variables, by utilizing the monotone operator splitting concept to develop algorithms that specifically suit the problem structure. The algorithm transforms a large-scale convex problem, at each iteration, into a set of much smaller subproblems (for instance, all subproblems are one-dimensional for the box-diagonal problems of extended linear-quadratic programming) that can be solved parallelly. The principle is one of splitting a monotone operator by expressing it as the sum of two simpler operators and then exploiting the resulting structure after analyzing the structure of problems. The suggested title of this splitting method is "forward-backward splitting" from the existing ordinary method. This methodology has the advantage of easy (or parallel) implementation and good decomposition. The numerical experiments of our methods and comparisons with some methods currently used show that our methods can fully utilize the decomposition structure of the problems we test and also that they converge much faster than the interior point method or the existing MPC toolbox in MATLAB for the large-scale cases of those problems. These numerical tests show the potential ability of the FBS methods to solve large-scale optimization problems.