On the complexity of piecewise affine system identification

The paper provides results regarding the computational complexity of hybrid system identification. More precisely, we focus on the estimation of piecewise affine (PWA) maps from input-output data and analyze the complexity of computing a global minimizer of the error. Previous work showed that a global solution could be obtained for continuous PWA maps with a worst-case complexity exponential in the number of data. In this paper, we show how global optimality can be reached for a slightly more general class of possibly discontinuous PWA maps with a complexity only polynomial in the number of data, however with an exponential complexity with respect to the data dimension. This result is obtained via an analysis of the intrinsic classification subproblem of associating the data points to the different modes. In addition, we prove that the problem is NP-hard, and thus that the exponential complexity in the dimension is a natural expectation for any exact algorithm.

[1]  A. Garulli,et al.  A survey on switched and piecewise affine system identification , 2012 .

[2]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[3]  Pierre Hansen,et al.  NP-hardness of Euclidean sum-of-squares clustering , 2008, Machine Learning.

[4]  Laurent Bako,et al.  Identification of switched linear systems via sparse optimization , 2011, Autom..

[5]  René Vidal,et al.  Identification of Hybrid Systems: A Tutorial , 2007, Eur. J. Control.

[6]  Constantino M. Lagoa,et al.  A Sparsification Approach to Set Membership Identification of Switched Affine Systems , 2012, IEEE Transactions on Automatic Control.

[7]  S. Sastry,et al.  An algebraic geometric approach to the identification of a class of linear hybrid systems , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[8]  Le Thi Hoai An,et al.  A Difference of Convex Functions Algorithm for Switched Linear Regression , 2014, IEEE Transactions on Automatic Control.

[9]  Henrik Ohlsson,et al.  Identification of switched linear regression models using sum-of-norms regularization , 2013, Autom..

[10]  Gérard Bloch,et al.  Learning smooth models of nonsmooth functions via convex optimization , 2012, 2012 IEEE International Workshop on Machine Learning for Signal Processing.

[11]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[12]  Alberto Bemporad,et al.  Identification of piecewise affine systems via mixed-integer programming , 2004, Autom..

[13]  Fabien Lauer.  Estimating the probability of success of a simple algorithm for switched linear regression , 2013 .

[14]  Alberto Bemporad,et al.  A bounded-error approach to piecewise affine system identification , 2005, IEEE Transactions on Automatic Control.

[15]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[16]  Henrik Ohlsson,et al.  Compression Based Identification of PWA Systems , 2014 .

[17]  René Vidal,et al.  A continuous optimization framework for hybrid system identification , 2011, Autom..

[18]  Rajeev Alur,et al.  Precise piecewise affine models from input-output data , 2014, 2014 International Conference on Embedded Software (EMSOFT).

[19]  John N. Tsitsiklis,et al.  A survey of computational complexity results in systems and control , 2000, Autom..

[20]  W. P. M. H. Heemels,et al.  A Bayesian approach to identification of hybrid systems , 2004, IEEE Transactions on Automatic Control.

[21]  Lennart Ljung,et al.  System identification (2nd ed.): theory for the user , 1999 .

[22]  Stephen P. Boyd,et al.  Segmentation of ARX-models using sum-of-norms regularization , 2010, Autom..

[23]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[24]  Manfred Morari,et al.  A clustering technique for the identification of piecewise affine systems , 2001, Autom..