Distributed Design for Nuclear Norm Minimization of Linear Matrix Equations With Constraints

This article aims at a distributed design to minimize the nuclear norm (the sum of all singular values) under linear equality constraints over a multiagent network. The problem is reformulated as a distributed trace norm minimization problem by introducing substitutional variables. A distributed projected primal-dual algorithm is proposed for the reformulation. It is shown that the algorithm converges to an optimal solution with a rate of $\mathcal O(1/t)$. Numerical simulations on three classical problems, including linear matrix equality constraints, cardinality minimization, and low-rank matrix completion, are carried out for illustration.

[1]  Yiguang Hong,et al.  Distributed Continuous-Time Algorithm for Constrained Convex Optimizations via Nonsmooth Analysis Approach , 2015, IEEE Transactions on Automatic Control.

[2]  Gordon F. Royle,et al.  Algebraic Graph Theory , 2001, Graduate texts in mathematics.

[3]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[4]  Stephen P. Boyd,et al.  Rank minimization and applications in system theory , 2004, Proceedings of the 2004 American Control Conference.

[5]  Shengyuan Xu,et al.  Regularized Primal–Dual Subgradient Method for Distributed Constrained Optimization , 2016, IEEE Transactions on Cybernetics.

[6]  G. Papanicolaou,et al.  Array imaging using intensity-only measurements , 2010 .

[7]  Shaoshuai Mou,et al.  A distributed algorithm for efficiently solving linear equations and its applications (Special Issue JCW) , 2016, Syst. Control. Lett..

[8]  Asuman E. Ozdaglar,et al.  Distributed Subgradient Methods for Multi-Agent Optimization , 2009, IEEE Transactions on Automatic Control.

[9]  Shengyuan Xu,et al.  Distributed Primal–Dual Subgradient Method for Multiagent Optimization via Consensus Algorithms , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[10]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[11]  Shaoshuai Mou,et al.  Distributed Algorithm for Achieving Minimum l1 Norm Solutions of Linear Equation , 2018, 2018 Annual American Control Conference (ACC).

[12]  Lihua Xie,et al.  Distributed constrained optimal consensus of multi-agent systems , 2016, Autom..

[13]  Qingshan Liu,et al.  A Multi-Agent System With a Proportional-Integral Protocol for Distributed Constrained Optimization , 2017, IEEE Transactions on Automatic Control.

[14]  Asuman E. Ozdaglar,et al.  Distributed Alternating Direction Method of Multipliers , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[15]  Shaoshuai Mou,et al.  A Distributed Algorithm for Least Squares Solutions , 2019, IEEE Transactions on Automatic Control.

[16]  Ying Xiong.  Nonlinear Optimization , 2014 .

[17]  Junfeng Yang,et al.  Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization , 2012, Math. Comput..

[18]  Enrique Mallada,et al.  The Role of Convexity in Saddle-Point Dynamics: Lyapunov Function and Robustness , 2016, IEEE Transactions on Automatic Control.

[19]  Yang Liu,et al.  An Arrow-Hurwicz-Uzawa Type Flow as Least Squares Solver for Network Linear Equations , 2017, Autom..

[20]  Stephen P. Boyd,et al.  Log-det heuristic for matrix rank minimization with applications to Hankel and Euclidean distance matrices , 2003, Proceedings of the 2003 American Control Conference, 2003..

[21]  Xuan Wang,et al.  A distributed linear equation solver for least square solutions , 2017, 2017 IEEE 56th Annual Conference on Decision and Control (CDC).

[22]  Shaoshuai Mou,et al.  A distributed algorithm for solving a linear algebraic equation , 2015, 2013 51st Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[23]  Walter Willinger,et al.  Spatio-temporal compressive sensing and internet traffic matrices , 2009, SIGCOMM '09.

[24]  Qing Ling,et al.  EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimization , 2014, 1404.6264.

[25]  Nicolas Gillis,et al.  Low-Rank Matrix Approximation with Weights or Missing Data Is NP-Hard , 2010, SIAM J. Matrix Anal. Appl..

[26]  Nathan Srebro,et al.  Fast maximum margin matrix factorization for collaborative prediction , 2005, ICML.

[27]  P. Olver.  Nonlinear Systems , 2013 .

[28]  Weiyu Xu,et al.  Necessary and sufficient conditions for success of the nuclear norm heuristic for rank minimization , 2008, 2008 47th IEEE Conference on Decision and Control.

[29]  Shu Liang,et al.  Distributed Computation of Linear Matrix Equations: An Optimization Perspective , 2017, IEEE Transactions on Automatic Control.

[30]  Massimiliano Pontil,et al.  Convex multi-task feature learning , 2008, Machine Learning.

[31]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..

[32]  Xuan Wang,et al.  Improvement of a Distributed Algorithm for Solving Linear Equations , 2017, IEEE Transactions on Industrial Electronics.

[33]  Qing Ling,et al.  Decentralized low-rank matrix completion , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[34]  Brian D. O. Anderson,et al.  Network Flows That Solve Linear Equations , 2015, IEEE Transactions on Automatic Control.

[35]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..