Vector Space and Matrix Methods in Signal and System Theory

The tools, ideas, and insights from linear algebra, abstract algebra, and functional analysis can be extremely useful to signal processing and system theory in various areas of engineering, science, and social science including approximation, optimization, parameter identification, big data, etc. Indeed, many important ideas can be developed from the simple operator equation A x = b by considering it in a variety of ways. If x and if b are vectors from the same or, perhaps, different vector spaces and A is an operator, there are three interesting questions that can be asked which provide a setting for a broad study.

[1]  Arthur Albert,et al.  Regression and the Moore-Penrose Pseudoinverse , 2012 .

[2]  Simon Haykin,et al.  Neural Networks and Learning Machines , 2010 .

[3]  G. Strang.  Introduction to Linear Algebra , 1993 .

[4]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[5]  S. T. Alexander,et al.  Globally optimal rational approximation using homotopy continuation methods , 1992, IEEE Trans. Signal Process..

[6]  Ivan W. Selesnick,et al.  Reweighted Least Squares Design of Fir Filters , 1992, The Digital Signal Processing workshop.

[7]  O. Christensen.  An introduction to frames and Riesz bases , 2002 .

[8]  S. Thomas Alexander,et al.  A relationship between the recursive least squares update and homotopy continuation methods , 1991, IEEE Trans. Signal Process..

[9]  Arnold Neumaier,et al.  Solving Ill-Conditioned and Singular Linear Systems: A Tutorial on Regularization , 1998, SIAM Rev..

[10]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[11]  Ricardo Arturo Vargas.  Iterative design of lp digital filters , 2008 .

[12]  C. Burrus,et al.  Introduction to Wavelets and Wavelet Transforms: A Primer , 1997 .

[13]  C. R. Rao,et al.  Generalized Inverse of Matrices and its Applications , 1972 .

[14]  T. Moon,et al.  Mathematical Methods and Algorithms for Signal Processing , 1999 .

[15]  P. Halmos.  Finite-Dimensional Vector Spaces , 1960 .

[16]  Soo-Chang Pei,et al.  An introduction to discrete finite frames , 1997, IEEE Signal Process. Mag..

[17]  Gilbert Strang.  LINEAR ALGEBRA and Learning from Data First Edition MANUAL FOR INSTRUCTORS , 2019 .

[18]  Richard G. Baraniuk,et al.  Compressive Sensing , 2008, Computer Vision, A Reference Guide.

[19]  S. Waldron.  An Introduction to Finite Tight Frames , 2018 .

[20]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[21]  C. D. Meyer,et al.  Generalized inverses of linear transformations , 1979 .

[22]  A. J. Sieradski.  An Introduction to Topology and Homotopy , 1991 .

[23]  C. Sidney Burrus,et al.  Constrained least square design of FIR filters without specified transition bands , 1996, IEEE Trans. Signal Process..

[24]  R. Young,et al.  An introduction to nonharmonic Fourier series , 1980 .

[25]  I. Selesnick.  Introduction to Sparsity in Signal Processing * , 2012 .

[26]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[27]  R. Marks.  Introduction to Shannon Sampling and Interpolation Theory , 1990 .

[28]  Audra E. Kosh,et al.  Linear Algebra and its Applications , 1992 .

[29]  R. E. Kalman,et al.  Linear system theory-The state space approach , 1965 .

[30]  I. Daubechies,et al.  Iteratively reweighted least squares minimization for sparse recovery , 2008, 0807.0575.

[31]  Adi Ben-Israel,et al.  Generalized inverses: theory and applications , 1974 .

[32]  R. Penrose.  On best approximate solutions of linear matrix equations , 1956, Mathematical Proceedings of the Cambridge Philosophical Society.

[33]  D. Luenberger.  Optimization by Vector Space Methods , 1968 .

[34]  J. Kovacevic,et al.  Life Beyond Bases: The Advent of Frames (Part I) , 2007, IEEE Signal Processing Magazine.

[35]  Soo-Chang Pei,et al.  Sparse Fast Fourier Transform by downsampling , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[36]  Bhaskar D. Rao,et al.  Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm , 1997, IEEE Trans. Signal Process..

[37]  Charles L. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[38]  Piotr Indyk,et al.  Nearly optimal sparse fourier transform , 2012, STOC '12.

[39]  J. Tinsley Oden,et al.  Applied functional analysis , 1996 .

[40]  R. Penrose.  A Generalized inverse for matrices , 1955 .

[41]  C. Sidney Burrus,et al.  Iterative reweighted least-squares design of FIR filters , 1994, IEEE Trans. Signal Process..

[42]  E. Cheney.  Introduction to approximation theory , 1966 .

[43]  M. Zwaan.  An introduction to hilbert space , 1990 .

[44]  Kannan Ramchandran,et al.  Computing a k-sparse n-length Discrete Fourier Transform using at most 4k samples and O(k log k) complexity , 2013, 2013 IEEE International Symposium on Information Theory.

[45]  C. Sidney Burrus,et al.  Constrained least squares design of FIR filters using iterative reweighted least squares , 1998, 9th European Signal Processing Conference (EUSIPCO 1998).

[46]  P. Halmos.  Introduction to Hilbert Space: And the Theory of Spectral Multiplicity , 1998 .

[47]  I. Daubechies.  Ten Lectures on Wavelets , 1992 .

[48]  Piotr Indyk,et al.  Simple and practical algorithm for sparse Fourier transform , 2012, SODA.

[49]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[50]  Cleve B. Moler,et al.  Numerical computing with MATLAB , 2004 .

[51]  D. Donoho.  For most large underdetermined systems of equations, the minimal 𝓁1‐norm near‐solution approximates the sparsest near‐solution , 2006 .

[52]  D. Donoho.  Unconditional Bases Are Optimal Bases for Data Compression and for Statistical Estimation , 1993 .

[53]  Gene H. Golub,et al.  Matrix computations , 1983 .

[54]  Ivan W. Selesnick,et al.  Introduction to Sparsity in Signal Processing 1 , 2012 .

[55]  E. Polak.  Introduction to linear and nonlinear programming , 1973 .

[56]  J. Navarro-Pedreño.  Numerical Methods for Least Squares Problems , 1996 .

[57]  A. Benveniste,et al.  Multiscale system theory , 1990, 29th IEEE Conference on Decision and Control.

[58]  C. Sidney Burrus,et al.  On the structure of efficient DFT algorithms , 1983, IEEE Trans. Acoust. Speech Signal Process..

[59]  Yonina C. Eldar.  Sampling Theory: Beyond Bandlimited Systems , 2015 .

[60]  Vivek K Goyal,et al.  Foundations of Signal Processing , 2014 .

[61]  Michael Athans.  Time-domain analysis and design of control systems , 1966 .

[62]  James S. Walker.  Fast Fourier Transforms , 1991 .

[63]  J. Kovacevic,et al.  Life Beyond Bases: The Advent of Frames (Part II) , 2007, IEEE Signal Processing Magazine.

[64]  C. S. Burrus,et al.  Least p-power error design of FIR filters , 1992, [Proceedings] 1992 IEEE International Symposium on Circuits and Systems.

[65]  Paulo J. S. G. Ferreira,et al.  Mathematics for Multimedia Signal Processing II: Discrete Finite Frames and Signal Reconstruction , 1999 .

[66]  K. S. Banerjee.  Generalized Inverse of Matrices and Its Applications , 1973 .