Applications of convex optimization in signal processing and digital communication

Abstract. In the last two decades, the mathematical programming community has witnessed some spectacular advances in interior point methods and robust optimization. These advances have recently started to significantly impact various fields of applied sciences and engineering where computational efficiency is essential. This paper focuses on two such fields: digital signal processing and communication. In the past, the widely used optimization methods in both fields had been the gradient descent or least squares methods, both of which are known to suffer from the usual headaches of stepsize selection, algorithm initialization and local minima. With the recent advances in conic and robust optimization, the opportunity is ripe to use the newly developed interior point optimization techniques and highly efficient software tools to help advance the fields of signal processing and digital communication. This paper surveys recent successes of applying interior point and robust optimization to solve some core problems in these two fields. The successful applications considered in this paper include adaptive filtering, robust beamforming, design and analysis of multi-user communication system, channel equalization, decoding and detection. Throughout, our emphasis is on how to exploit the hidden convexity, convex reformulation of semi-infinite constraints, analysis of convergence, complexity and performance, as well as efficient practical implementation.

[1]  Kenneth Steiglitz,et al.  METEOR: a constraint-based FIR filter design program , 1992, IEEE Trans. Signal Process..

[2]  Stephen Boyd,et al.  On achieving reduced error propagation sensitivity in DFE design via convex optimization , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[3]  Valerii Fedorov,et al.  Design of Experiments in Statistics , 2000 .

[4]  Brian D. O. Anderson,et al.  Recursive algorithm for spectral factorization , 1974 .

[5]  Wei Yu,et al.  Sum capacity of Gaussian vector broadcast channels , 2004, IEEE Transactions on Information Theory.

[6]  P. P. Vaidyanathan,et al.  Globally optimal two channel FIR orthonormal filter banks adapted to the input signal statistics , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[7]  Arkadi Nemirovski,et al.  Robust Convex Optimization , 1998, Math. Oper. Res..

[8]  Zhi-Quan Luo,et al.  An efficient quasi-maximum likelihood decoder for PSK signals , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[9]  Zhi-Quan Luo,et al.  Quasi-maximum-likelihood multiuser detection using semi-definite relaxation with application to synchronous CDMA , 2002, IEEE Trans. Signal Process..

[10]  L. Vandenberghe,et al.  Handling nonnegative constraints in spectral estimation , 2000, Conference Record of the Thirty-Fourth Asilomar Conference on Signals, Systems and Computers (Cat. No.00CH37154).

[11]  E. Yaz.  Linear Matrix Inequalities In System And Control Theory , 1998, Proceedings of the IEEE.

[12]  Kim-Chuan Toh,et al.  SDPT3 -- A Matlab Software Package for Semidefinite Programming , 1996 .

[13]  Yin Zhang,et al.  Rank-Two Relaxation Heuristics for MAX-CUT and Other Binary Quadratic Programs , 2002, SIAM J. Optim..

[14]  Stephen P. Boyd,et al.  Optimizing dominant time constant in RC circuits , 1998, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[15]  LiWu Chang,et al.  Performance of DMI and eigenspace-based beamformers , 1992 .

[16]  Emre Telatar,et al.  Capacity of Multi-antenna Gaussian Channels , 1999, Eur. Trans. Telecommun..

[17]  Zhi Ding,et al.  A fast linear programming algorithm for blind equalization , 2000, IEEE Trans. Commun..

[18]  Laurent El Ghaoui,et al.  Robust Solutions to Least-Squares Problems with Uncertain Data , 1997, SIAM J. Matrix Anal. Appl..

[19]  Randy L. Haupt,et al.  Introduction to Adaptive Arrays , 1980 .

[20]  Georgios B. Giannakis,et al.  Wireless multicarrier communications , 2000, IEEE Signal Process. Mag..

[21]  Andreas Antoniou,et al.  Constrained minimum-BER multiuser detection , 2000, IEEE Trans. Signal Process..

[22]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[23]  Andrea J. Goldsmith,et al.  Capacity and optimal power allocation for fading broadcast channels with minimum rates , 2001, GLOBECOM'01. IEEE Global Telecommunications Conference (Cat. No.01CH37270).

[24]  Z. Luo,et al.  Robust array interpolation using second-order cone programming , 2002, IEEE Signal Processing Letters.

[25]  C. Sidney Burrus,et al.  Constrained least square design of FIR filters without specified transition bands , 1996, IEEE Trans. Signal Process..

[26]  Andreas Antoniou,et al.  A near-optimal multiuser detector for CDMA channels using semidefinite programming relaxation , 2001, ISCAS 2001. The 2001 IEEE International Symposium on Circuits and Systems (Cat. No.01CH37196).

[27]  Zhi-Quan Luo,et al.  Design of robust IIR magnitude filters via semidefinite programming , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[28]  Zhi-Quan Luo,et al.  Multi-carrier multiple access is sum-rate optimal for block transmissions over circulant ISI channels , 2002, 2002 IEEE International Conference on Communications. Conference Proceedings. ICC 2002 (Cat. No.02CH37333).

[29]  Henry Cox,et al.  Robust adaptive beamforming , 2005, IEEE Trans. Acoust. Speech Signal Process..

[30]  D. Bertsimas,et al.  Moment Problems and Semidefinite Optimization , 2000 .

[31]  Stephen P. Boyd,et al.  FIR filter design via semidefinite programming and spectral factorization , 1996, Proceedings of 35th IEEE Conference on Decision and Control.

[32]  U. Fincke,et al.  Improved methods for calculating vectors of short length in a lattice , 1985 .

[33]  Yurii Nesterov,et al.  Squared Functional Systems and Optimization Problems , 2000 .

[34]  Zhi-Quan Luo,et al.  Blind constant modulus equalization via convex optimization , 2003, IEEE Trans. Signal Process..

[35]  Zhi-Quan Luo,et al.  Linear matrix inequality formulation of spectral mask constraints , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[36]  K. Teo,et al.  A new approach to the optimization of robust antenna array processors , 1993 .

[37]  Jian-Kang Zhang,et al.  A fractionally spaced blind equalizer based on linear programming , 2002, IEEE Trans. Signal Process..

[38]  Stephen P. Boyd,et al.  Applications of second-order cone programming , 1998 .

[39]  Stephen P. Boyd,et al.  Semidefinite Programming , 1996, SIAM Rev..

[40]  B. Hassibi,et al.  On the expected complexity of sphere decoding , 2001, Conference Record of Thirty-Fifth Asilomar Conference on Signals, Systems and Computers (Cat.No.01CH37256).

[41]  G. Iyengar,et al.  TR-2002-04 Robust convex quadratically constrained programs ∗ , 2002 .

[42]  Eric Moulines,et al.  Semidefinite positive relaxation of the maximum-likelihood criterion applied to multiuser detection in a CDMA context , 2002, IEEE Signal Processing Letters.

[43]  Lars K. Rasmussen,et al.  Constrained maximum-likelihood detection in CDMA , 2001, IEEE Trans. Commun..

[44]  Zhi-Quan Luo,et al.  Minimum BER block precoders for zero-forcing equalization , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[45]  David P. Williamson,et al.  Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming , 1995, JACM.

[46]  Stephen P. Boyd,et al.  Joint optimization of communication rates and linear systems , 2001, Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No.01CH37228).

[47]  Petre Stoica,et al.  MA estimation in polynomial time , 2000, IEEE Trans. Signal Process..

[48]  R. Saigal,et al.  Handbook of semidefinite programming : theory, algorithms, and applications , 2000 .

[49]  T. W. Parks,et al.  Digital Filter Design , 1987 .

[50]  O. Herrmann,et al.  Design of nonrecursive digital filters with minimum phase , 1970 .

[51]  Zhi Ding,et al.  Robust blind multiuser detection against signature waveform mismatch based on second-order cone programming , 2005, IEEE Transactions on Wireless Communications.

[52]  Zhi-Quan Luo,et al.  Robust adaptive beamforming for general-rank signal models , 2003, IEEE Trans. Signal Process..

[53]  Stephen P. Boyd,et al.  sdpsol: a parse/solver for semidefinite programs with matrix structure , 1999 .

[54]  Petre Stoica,et al.  On the parameterization of positive real sequences and MA parameter estimation , 2001, IEEE Trans. Signal Process..

[55]  G. Giannakis,et al.  Wireless Multicarrier Communications where Fourier Meets , 2022 .

[56]  Stephen P. Boyd,et al.  Antenna array pattern synthesis via convex optimization , 1997, IEEE Trans. Signal Process..

[57]  Stephen P. Boyd,et al.  FIR Filter Design via Spectral Factorization and Convex Optimization , 1999 .

[58]  I. Reed,et al.  Rapid Convergence Rate in Adaptive Arrays , 1974, IEEE Transactions on Aerospace and Electronic Systems.

[59]  Yinyu Ye,et al.  Application of Semidefinite Programming to Circuit Partitioning , 2000 .

[60]  Hans D. Mittelmann,et al.  An independent benchmarking of SDP and SOCP solvers , 2003, Math. Program..

[61]  Zhi-Quan Luo,et al.  Interior point least squares estimation: transient convergence analysis and application to MMSE decision-feedback equalization , 2001, IEEE Trans. Signal Process..

[62]  Zhi-Quan Luo,et al.  Robust adaptive beamforming using worst-case performance optimization: a solution to the signal mismatch problem , 2003, IEEE Trans. Signal Process..

[63]  Zhi-Quan Luo,et al.  Robust Filtering via Semidefinite Programming with Applications to Target Tracking , 2002, SIAM J. Optim..

[64]  Emanuele Viterbo,et al.  A universal lattice code decoder for fading channels , 1999, IEEE Trans. Inf. Theory.

[65]  Arkadi Nemirovski,et al.  Robust optimization – methodology and applications , 2002, Math. Program..

[66]  Zhi-Quan Luo,et al.  Transceiver optimization for block-based multiple access through ISI channels , 2004, IEEE Transactions on Signal Processing.

[67]  Timothy N. Davidson,et al.  Efficient design of waveforms for robust pulse amplitude modulation using mean square error criteria , 2000, 2000 10th European Signal Processing Conference.

[68]  Yinyu Ye,et al.  Interior point algorithms: theory and analysis , 1997 .

[69]  Andrea J. Goldsmith,et al.  Duality, achievable rates, and sum-rate capacity of Gaussian MIMO broadcast channels , 2003, IEEE Trans. Inf. Theory.

[70]  Yinyu Ye,et al.  A .699-Approximation Algorithm for Max-Bisection , 1999 .

[71]  Ning Ma,et al.  Adaptive beamforming with joint robustness against mismatched signal steering vector and interference nonstationarity , 2004, IEEE Signal Processing Letters.

[72]  Zhi-Quan Luo,et al.  Adaptive linear filtering using interior point optimization techniques , 2000, IEEE Trans. Signal Process..

[73]  Zhi-Quan Luo,et al.  Multivariate Nonnegative Quadratic Mappings , 2003, SIAM J. Optim..

[74]  Andrea J. Goldsmith,et al.  On the duality of Gaussian multiple-access and broadcast channels , 2002, IEEE Transactions on Information Theory.

[75]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[76]  Stephen P. Boyd,et al.  Robust minimum variance beamforming , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[77]  Arkadi Nemirovski,et al.  Lectures on modern convex optimization - analysis, algorithms, and engineering applications , 2001, MPS-SIAM series on optimization.

[78]  Lieven Vandenberghe,et al.  Interior-point methods for magnitude filter design , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[79]  Wei Yu,et al.  Multiuser detection for vector multiple access channels using generalized decision feedback equalization , 2000, WCC 2000 - ICSP 2000. 2000 5th International Conference on Signal Processing Proceedings. 16th World Computer Congress 2000.

[80]  Stephen P. Boyd,et al.  Linear Matrix Inequalities in Systems and Control Theory , 1994 .

[81]  K. M. Wong,et al.  Adaptive beamforming with sidelobe control: a second-order cone programming approach , 2003, IEEE Signal Processing Letters.

[82]  T W Parks,et al.  Design of optimal minimum phase FIR filters by direct factorization , 1986 .

[83]  William J. Cook,et al.  Combinatorial optimization , 1997 .

[84]  Gerard J. Foschini,et al.  Layered space-time architecture for wireless communication in a fading environment when using multi-element antennas , 1996, Bell Labs Technical Journal.

[85]  Kok Lay Teo,et al.  The dual parameterization approach to optimal least square FIR filter design subject to maximum error constraints , 2000, IEEE Trans. Signal Process..

[86]  Babak Hassibi,et al.  Cayley differential unitary space - Time codes , 2002, IEEE Trans. Inf. Theory.

[87]  Laurent El Ghaoui,et al.  Robust Solutions to Uncertain Semidefinite Programs , 1998, SIAM J. Optim..

[88]  Z. Luo,et al.  Finite horizon robust Kalman filter design , 1999, Proceedings of the 38th IEEE Conference on Decision and Control (Cat. No.99CH36304).

[89]  M. R. Rao,et al.  Combinatorial Optimization , 1992, NATO ASI Series.