Constrained Optimization with Low-Rank Tensors and Applications to Parametric Problems with PDEs

Low-rank tensor methods provide efficient representations and computations for high-dimensional problems and are able to break the curse of dimensionality when dealing with systems involving multiple parameters. Motivated by optimal control problems with PDEs under uncertainty, we present algorithms for constrained nonlinear optimization problems that use low-rank tensors. They are applied to optimal control of PDEs with uncertain parameters and to parametrized variational inequalities of obstacle type. These methods are tailored to the usage of low-rank tensor arithmetics and allow us to solve huge scale optimization problems. In particular, we consider a semismooth Newton method for an optimal control problem with pointwise control constraints and an interior point algorithm for an obstacle problem, both with uncertainties in the coefficients.

[1]  W. Hackbusch,et al.  A New Scheme for the Tensor Representation , 2009 .

[2]  Robert E. Mahony,et al.  Optimization Algorithms on Matrix Manifolds , 2007 .

[3]  Homer F. Walker,et al.  Globally Convergent Inexact Newton Methods , 1994, SIAM J. Optim..

[4]  Michael Steinlechner,et al.  Riemannian Optimization for High-Dimensional Tensor Completion , 2016, SIAM J. Sci. Comput..

[5]  Daniel Kressner,et al.  Preconditioned Low-rank Riemannian Optimization for Linear Systems with Tensor Product Structure , 2015, SIAM J. Sci. Comput..

[6]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[7]  André Uschmajew,et al.  On Local Convergence of Alternating Schemes for Optimization of Convex Problems in the Tensor Train Format , 2013, SIAM J. Numer. Anal..

[8]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[9]  Ivan V. Oseledets,et al.  DMRG Approach to Fast Linear Algebra in the TT-Format , 2011, Comput. Methods Appl. Math..

[10]  Daniel Kressner,et al.  Algorithm 941 , 2014 .

[11]  Antonio Falcó,et al.  Proper generalized decomposition for nonlinear convex problems in tensor Banach spaces , 2011, Numerische Mathematik.

[12]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[13]  Drew P. Kouri,et al.  Risk-Averse PDE-Constrained Optimization Using the Conditional Value-At-Risk , 2016, SIAM J. Optim..

[14]  Ralf Kornhuber,et al.  A polynomial chaos approach to stochastic variational inequalities , 2010, J. Num. Math..

[15]  Luís N. Vicente,et al.  Analysis of Inexact Trust-Region SQP Algorithms , 2002, SIAM J. Optim..

[16]  D. Kinderlehrer,et al.  An introduction to variational inequalities and their applications , 1980 .

[17]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[18]  Arnold Neumaier,et al.  Global Optimization by Multilevel Coordinate Search , 1999, J. Glob. Optim..

[19]  Stefan Ulbrich,et al.  Adaptive Multilevel Inexact SQP Methods for PDE-Constrained Optimization , 2011, SIAM J. Optim..

[20]  Liqun Qi,et al.  A nonsmooth version of Newton's method , 1993, Math. Program..

[21]  Lars Grasedyck,et al.  Hierarchical Singular Value Decomposition of Tensors , 2010, SIAM J. Matrix Anal. Appl..

[22]  Pierre Comon,et al.  Canonical Polyadic Decomposition with a Columnwise Orthonormal Factor Matrix , 2012, SIAM J. Matrix Anal. Appl..

[23]  Daniel Kressner,et al.  Low-Rank Tensor Krylov Subspace Methods for Parametrized Linear Systems , 2011, SIAM J. Matrix Anal. Appl..

[24]  Boris N. Khoromskij,et al.  Approximate iterations for structured matrices , 2008, Numerische Mathematik.

[25]  Harald Niederreiter,et al.  Random number generation and Quasi-Monte Carlo methods , 1992, CBMS-NSF regional conference series in applied mathematics.

[26]  S. V. Dolgov,et al.  ALTERNATING MINIMAL ENERGY METHODS FOR LINEAR SYSTEMS IN HIGHER DIMENSIONS∗ , 2014 .

[27]  Ivan Oseledets,et al.  Tensor-Train Decomposition , 2011, SIAM J. Sci. Comput..

[28]  Hermann G. Matthies,et al.  Efficient Analysis of High Dimensional Data in Tensor Formats , 2012 .

[29]  Daniel Kressner,et al.  A literature survey of low‐rank tensor approximation techniques , 2013, 1302.7121.

[30]  RWTH Aachen,et al.  Adaptive Low-Rank Methods for Problems on Sobolev Spaces with Error Control in $L_2$ , 2014, 1412.3951.

[31]  Frances Y. Kuo,et al.  Multi-level quasi-Monte Carlo finite element methods for a class of elliptic partial differential equations with random coefficients , 2012, 1208.6349.

[32]  F. L. Hitchcock.  The Expression of a Tensor or a Polyadic as a Sum of Products , 1927 .

[33]  Jochen Garcke,et al.  Sparse Grids in a Nutshell , 2012 .

[34]  Reinhold Schneider,et al.  The Alternating Linear Scheme for Tensor Optimization in the Tensor Train Format , 2012, SIAM J. Sci. Comput..

[35]  E. Tyrtyshnikov,et al.  TT-cross approximation for multidimensional arrays , 2010 .

[36]  Kazufumi Ito,et al.  The Primal-Dual Active Set Strategy as a Semismooth Newton Method , 2002, SIAM J. Optim..

[37]  Bart Vandereycken,et al.  Low-rank tensor completion by Riemannian optimization , 2014 .

[38]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[39]  Matthias Heinkenschloss,et al.  Inexact Objective Function Evaluations in a Trust-Region Algorithm for PDE-Constrained Optimization under Uncertainty , 2014, SIAM J. Sci. Comput..

[40]  Lars Grasedyck,et al.  Hierarchical Tensor Approximation of Output Quantities of Parameter-Dependent PDEs , 2015, SIAM/ASA J. Uncertain. Quantification.

[41]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[42]  Bart Vandereycken,et al.  The geometry of algorithms using hierarchical tensors , 2013, Linear Algebra and its Applications.

[43]  Felix J. Herrmann,et al.  Optimization on the Hierarchical Tucker manifold – Applications to tensor completion , 2014, Linear Algebra and its Applications.

[44]  Michael Ulbrich,et al.  Semismooth Newton Methods for Operator Equations in Function Spaces , 2002, SIAM J. Optim..

[45]  Hermann G. Matthies,et al.  Efficient low-rank approximation of the stochastic Galerkin matrix in tensor formats , 2014, Comput. Math. Appl..

[46]  T. Schulte-Herbrüggen,et al.  Computations in quantum tensor networks , 2012, 1212.5005.

[47]  W. Hackbusch.  Tensor Spaces and Numerical Tensor Calculus , 2012, Springer Series in Computational Mathematics.

[48]  Wolfgang Dahmen,et al.  Adaptive Near-Optimal Rank Tensor Approximation for High-Dimensional Operator Equations , 2013, Foundations of Computational Mathematics.

[49]  Andrea Barth,et al.  Multi-level Monte Carlo Finite Element method for elliptic PDEs with stochastic coefficients , 2011, Numerische Mathematik.

[50]  Michael Ulbrich,et al.  Semismooth Newton Methods for Variational Inequalities and Constrained Optimization Problems in Function Spaces , 2011, MOS-SIAM Series on Optimization.

[51]  Ors Legeza,et al.  Simulating strongly correlated quantum systems with tree tensor networks , 2010, 1006.3095.

[52]  Reinhold Schneider,et al.  Tensor Networks and Hierarchical Tensors for the Solution of High-Dimensional Partial Differential Equations , 2016, Foundations of Computational Mathematics.

[53]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[54]  Stefan Ulbrich,et al.  Optimization with PDE Constraints , 2008, Mathematical modelling.

[55]  S. Bellavia.  Inexact Interior-Point Method , 1998 .