A Convex Optimization Approach to Generalized Moment Problems

In this paper we present a universal solution to the generalized moment problem, with a nonclassical complexity constraint. We show that this solution can be obtained by minimizing a strictly convex nonlinear functional. This optimization problem is derived in two different ways. We first derive this intrinsically, in a geometric way, by path integration of a one-form which defines the generalized moment problem. It is observed that this one-form is closed and defined on a convex set, and thus exact with, perhaps surprisingly, a strictly convex primitive function. We also derive this convex functional as the dual problem of a problem to maximize a cross entropy functional. In particular, these approaches give a constructive parameterization of all solutions to the Nevanlinna-Pick interpolation problem, with possible higher-order interpolation at certain points in the complex plane, with a degree constraint as well as all solutions to the rational covariance extension problem — two areas which have been advanced by the work of Hidenori Kimura. Illustrations of these results in system identification and probability are also mentioned.

[1]  C. Byrnes,et al.  A Convex Optimization Approach to the Rational Covariance Extension Problem , 1999 .

[2]  Tryphon T. Georgiou,et al.  A generalized entropy criterion for Nevanlinna-Pick interpolation with degree constraint , 2001, IEEE Trans. Autom. Control..

[3]  H. Kimura.  Directional interpolation in the state space , 1988 .

[4]  Tryphon T. Georgiou,et al.  Realization of power spectra from partial covariance sequences , 1987, IEEE Trans. Acoust. Speech Signal Process..

[5]  C. Byrnes,et al.  Generalized interpolation in $H^\infty$ with a complexity constraint , 2004 .

[6]  Y. Genin,et al.  On the role of the Nevanlinna–Pick problem in circuit and system theory† , 1981 .

[7]  Tryphon T. Georgiou,et al.  A new approach to spectral estimation: a tunable high-resolution spectral estimator , 2000, IEEE Trans. Signal Process..

[8]  P. Enqvist.  Spectral Estimation by Geometric, Topological and Optimization Methods , 2001 .

[9]  Ryozo Nagamune.  Closed-loop shaping based on Nevanlinna-Pick interpolation with a degree bound , 2004, IEEE Transactions on Automatic Control.

[10]  Ryozo Nagamune,et al.  Sensitivity shaping in feedback control and analytic interpolation theory , 2001 .

[11]  J. Bokor,et al.  Minimal partial realization from orthonormal basis function expansions , 2001, Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No.01CH37228).

[12]  Anders Lindquist,et al.  Identifiability and Well-Posedness of Shaping-Filter Parameterizations: A Global Analysis Approach , 2002, SIAM J. Control. Optim..

[13]  H. Kimura,et al.  State space approach to the classical interpolation problem and its applications , 1989 .

[14]  Zoltán Szabó,et al.  Extended Ho-Kalman algorithm for systems represented in generalized orthonormal bases , 2000, Autom..

[15]  J. Shohat,et al.  The problem of moments , 1943 .

[16]  J. Doyle,et al.  Essentials of Robust Control , 1997 .

[17]  Bruce A. Francis,et al.  Feedback Control Theory , 1992 .

[18]  R. Kálmán.  Realization of Covariance Sequences , 1982 .

[19]  Anders Lindquist,et al.  On the Duality between Filtering and Nevanlinna--Pick Interpolation , 2000, SIAM J. Control. Optim..

[20]  Hidenori Kimura,et al.  Directional interpolation approach to H ∞ -Optimization and robust stabilization , 1987 .

[21]  Anders Lindquist,et al.  From Finite Covariance Windows to Modeling Filters: A Convex Optimization Approach , 2001, SIAM Rev..

[22]  Anders Lindquist,et al.  Geometry of the Kimura-Georgiou parametrization of modelling filters , 1989 .

[23]  A. Lindquist.  A New Algorithm for Optimal Filtering of Discrete-Time Stationary Processes , 1974 .

[24]  A. Tannenbaum.  Feedback stabilization of linear dynamical plants with uncertainty in the gain factor , 1980 .

[25]  Imre Csiszár,et al.  Information projections revisited , 2000, IEEE Trans. Inf. Theory.

[26]  C. Byrnes,et al.  A complete parameterization of all positive rational extensions of a covariance sequence , 1995, IEEE Trans. Autom. Control..

[27]  U. Grenander,et al.  Toeplitz Forms And Their Applications , 1958 .

[28]  S. Kullback,et al.  Information Theory and Statistics , 1959 .

[29]  Tryphon T. Georgiou,et al.  The interpolation problem with a degree constraint , 1999, IEEE Trans. Autom. Control..

[30]  M. G. Kreĭn,et al.  Some questions in the theory of moments , 1962 .

[31]  H. Kimura.  Robust stabilizability for a class of transfer functions , 1983, The 22nd IEEE Conference on Decision and Control.

[32]  T. Georgiou.  A topological approach to Nevanlinna-pick interpolation , 1987 .

[33]  Predictability and Unpredictability in Kalman Filtering , 2004 .

[34]  H. Kimura.  Conjugation, interpolation and model-matching in H ∞ , 1989 .

[35]  A. Lindquist.  Some reduced-order non-Riccati equations for linear least-squares estimation : the stationary, single-output case† , 1976 .

[36]  W. Rudin.  Real and complex analysis , 1968 .

[37]  Anders Lindquist,et al.  Cepstral coefficients, covariance lags, and pole-zero models for finite data strings , 2001, IEEE Trans. Signal Process..

[38]  B. Wahlberg.  System identification using Kautz models , 1994, IEEE Trans. Autom. Control..

[39]  Anders Lindquist,et al.  On the Nonlinear Dynamics of Fast Filtering Algorithms , 1994 .

[40]  B. Wahlberg.  System identification using Laguerre models , 1991 .

[41]  I. Good.  Maximum Entropy for Hypothesis Formulation, Especially for Multidimensional Contingency Tables , 1963 .

[42]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[43]  I. Csiszár.  $I$-Divergence Geometry of Probability Distributions and Minimization Problems , 1975 .

[44]  J. William Helton,et al.  Non-Euclidean functional analysis and electronics , 1982 .

[45]  Arjan van der Schaft,et al.  Three decades of mathematical system theory , 1989 .

[46]  Ryozo Nagamune.  A robust solver using a continuation method for Nevanlinna-Pick interpolation with degree constraint , 2003, IEEE Trans. Autom. Control..

[47]  M. Kreĭn,et al.  The Markov Moment Problem and Extremal Problems , 1977 .

[48]  Ryozo Nagamune.  Robust Control with Complexity Constraint : A Nevanlinna-Pick Interpolation Approach , 2002 .

[49]  Paul Van Dooren,et al.  Speech modelling and the trigonometric moment problem , 1982 .