An SDP approach for l0-minimization: Application to ARX model segmentation

Minimizing the @?"0-seminorm of a vector under convex constraints is a combinatorial (NP-hard) problem. Replacement of the @?"0-seminorm with the @?"1-norm is a commonly used approach to compute an approximate solution of the original @?"0-minimization problem by means of convex programming. In the theory of compressive sensing, the condition that the sensing matrix satisfies the Restricted Isometry Property (RIP) is a sufficient condition to guarantee that the solution of the @?"1-approximated problem is equal to the solution of the original @?"0-minimization problem. However, the evaluation of the conservativeness of the @?"1-relaxation approaches is recognized to be a difficult task in case the RIP is not satisfied. In this paper, we present an alternative approach to minimize the @?"0-norm of a vector under given constraints. In particular, we show that an @?"0-minimization problem can be relaxed into a sequence of semidefinite programming problems, whose solutions are guaranteed to converge to the optimizer (if unique) of the original combinatorial problem also in case the RIP is not satisfied. Segmentation of ARX models is then discussed in order to show, through a relevant problem in system identification, that the proposed approach outperforms the @?"1-based relaxation in detecting piece-wise constant parameter changes in the estimated model.

[1]  Beibei Guo,et al.  A segmented regime-switching model with its application to stock market indices , 2011 .

[2]  Jean B. Lasserre,et al.  Global Optimization with Polynomials and the Problem of Moments , 2000, SIAM J. Optim..

[3]  Ting Sun,et al.  Single-pixel imaging via compressive sampling , 2008, IEEE Signal Process. Mag..

[4]  Tamer Basar,et al.  Sparsity based feedback design: A new paradigm in opportunistic sensing , 2011, Proceedings of the 2011 American Control Conference.

[5]  Felix J. Herrmann,et al.  Non-parametric seismic data recovery with curvelet frames , 2008 .

[6]  Joel A. Tropp,et al.  Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit , 2006, Signal Process..

[7]  Rick Chartrand,et al.  Exact Reconstruction of Sparse Signals via Nonconvex Minimization , 2007, IEEE Signal Processing Letters.

[8]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[9]  S. Mallat,et al.  Adaptive greedy approximations , 1997 .

[10]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[11]  Joel A. Tropp,et al.  ALGORITHMS FOR SIMULTANEOUS SPARSE APPROXIMATION , 2006 .

[12]  Pablo A. Parrilo,et al.  Semidefinite programming relaxations for semialgebraic problems , 2003, Math. Program..

[13]  Håkan Hjalmarsson,et al.  Sparse estimation based on a validation criterion , 2011, IEEE Conference on Decision and Control and European Control Conference.

[14]  Stéphane Canu,et al.  Recovering Sparse Signals With a Certain Family of Nonconvex Penalties and DC Programming , 2009, IEEE Transactions on Signal Processing.

[15]  Jitendra Malik,et al.  Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[17]  Stephen P. Boyd,et al.  Segmentation of ARX-models using sum-of-norms regularization , 2010, Autom..

[18]  Li Wang,et al.  Regularization Methods for SDP Relaxations in Large-Scale Polynomial Optimization , 2009, SIAM J. Optim..

[19]  Peter Andersson,et al.  Adaptive Forgetting in Recursive Identification through Multiple Models , 1985 .

[20]  M. Kojima,et al.  B-411 Sums of Squares and Semidefinite Programming Relaxations for Polynomial Optimization Problems with Structured Sparsity , 2004 .

[21]  Graziano Chesi,et al.  Solving quadratic distance problems: an LMI-based approach , 2003, IEEE Trans. Autom. Control..

[22]  Masakazu Muramatsu,et al.  EQUALITY BASED CONTRACTION OF SEMIDEFINITE PROGRAMMING RELAXATIONS IN POLYNOMIAL OPTIMIZATION , 2008 .

[23]  Lennart Ljung,et al.  Regressor and structure selection in NARX models using a structured ANOVA approach , 2008, Autom..

[24]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[25]  Jean B. Lasserre,et al.  Convergent SDP-Relaxations in Polynomial Optimization with Sparsity , 2006, SIAM J. Optim..

[26]  Constantino M. Lagoa,et al.  A Sparsification Approach to Set Membership Identification of Switched Affine Systems , 2012, IEEE Transactions on Automatic Control.

[27]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[28]  Masakazu Muramatsu,et al.  Sums of Squares and Semidefinite Programming Relaxations for Polynomial Optimization Problems with Structured Sparsity , 2004 .

[29]  J. Lasserre,et al.  Detecting global optimality and extracting solutions in GloptiPoly , 2003 .

[30]  Tyrone L. Vincent,et al.  Concentration of measure inequalities for compressive toeplitz matrices with applications to detection and system identification , 2010, 49th IEEE Conference on Decision and Control (CDC).

[31]  M. Laurent.  Sums of Squares, Moment Matrices and Optimization Over Polynomials , 2009 .

[32]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[33]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[34]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[35]  J. Lasserre,et al.  Solving nonconvex optimization problems , 2004, IEEE Control Systems.

[36]  Alberto Bemporad,et al.  A bounded-error approach to piecewise affine system identification , 2005, IEEE Transactions on Automatic Control.

[37]  Daniel E. Quevedo,et al.  Sparse Representations for Packetized Predictive Networked Control , 2013, ArXiv.

[38]  Richard E. Grandy,et al.  Orlando, Florida, USA , 2011 .

[39]  E.J. Candes.  Compressive Sampling , 2022 .

[40]  Tyrone L. Vincent,et al.  Compressive System Identification in the Linear Time-Invariant framework , 2011, IEEE Conference on Decision and Control and European Control Conference.