A quadratically convergent primal decomposition algorithm with soft coupling for nonlinear parameter estimation

Distributed algorithms for sparse, large-scale optimization problems are preferable over centralized solvers when the computational units are physically far apart from each other or the problem size is too large for the available memory. However, most distributed methods sacrifice convergence speed for simpler computations. In this paper, we propose a novel algorithm for a certain class of nonconvex, separable optimization problems that combines both distributed computations and locally quadratic convergence. It is based on the principle of primal decomposition with exact Hessian information but uses soft coupling between the agents to avoid global calculations and adapt faster to online data changes. An important application field of the presented method is nonlinear parameter estimation, where increasing the number of experiments may lead to problem dimensions that are prohibitive for conventional solvers. We assess the performance of our method on the identification of an Airborne Wind Energy (AWE) system using real-world experimental data.

[1]  Carl D. Laird,et al.  An interior-point method for efficient solution of block-structured NLP problems using an implicit Schur-complement decomposition , 2014, Comput. Chem. Eng..

[2]  Victor M. Zavala,et al.  Clustering-based preconditioning for stochastic programs , 2016, Comput. Optim. Appl..

[3]  H. Bock,et al.  A Multiple Shooting Algorithm for Direct Solution of Optimal Control Problems , 1984 .

[4]  Moritz Diehl,et al.  A quaternion‐based model for optimal control of an airborne wind energy system , 2015, ArXiv.

[5]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[6]  Joel Andersson,et al.  A General-Purpose Software Framework for Dynamic Optimization (Een algemene softwareomgeving voor dynamische optimalisatie) , 2013 .

[7]  D. Bertsekas Convexification procedures and decomposition methods for nonconvex optimization problems , 1979 .

[8]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[9]  Anthony V. Fiacco,et al.  Introduction to Sensitivity and Stability Analysis in Nonlinear Programming , 2012 .

[10]  Paul T. Boggs,et al.  Sequential Quadratic Programming , 1995, Acta Numerica.

[11]  Quoc Tran Dinh Sequential Convex Programming and Decomposition Approaches for Nonlinear Optimization , 2012 .

[12]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[13]  H. Bock,et al.  Recent Advances in Parameteridentification Techniques for O.D.E. , 1983 .

[14]  Dimitri P. Bertsekas,et al.  Incremental proximal methods for large scale convex optimization , 2011, Math. Program..

[15]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[16]  R. Kass,et al.  Approximate Bayesian Inference in Conditionally Independent Hierarchical Models (Parametric Empirical Bayes Models) , 1989 .

[17]  Abdelouahed Hamdi,et al.  Two-level primal-dual proximal decomposition technique to solve large scale optimization problems , 2005, Appl. Math. Comput..

[18]  Lorenz T. Biegler,et al.  On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming , 2006, Math. Program..

[19]  Shih-Ping Han A parallel algorithm for a class of convex programs , 1988 .

[20]  Jacek Gondzio,et al.  Exploiting structure in parallel implementation of interior point methods for optimization , 2009, Comput. Manag. Sci..

[21]  Victor M. Zavala,et al.  Interior-point decomposition approaches for parallel solution of large-scale nonlinear parameter estimation problems , 2008 .

[22]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[23]  Michael Erhard,et al.  Theory and Experimental Validation of a Simple Comprehensible Model of Tethered Kite Dynamics Used for Controller Design , 2013 .

[24]  Moritz Diehl,et al.  Autogenerating microsecond solvers for nonlinear MPC: A tutorial using ACADO integrators , 2015 .

[25]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.