Wavelet footprints and frames for signal processing and communication

Linear transforms and expansions are fundamental mathematical tools of signal processing. In particular, the wavelet transform has played an important role in several signal processing tasks, compression being a prime example. A signal can be represented in a basis or a frame. Frames, which are an extension of bases to overcomplete sets of vectors, are sometimes preferred to bases because of their greater design freedom and recently they have had an important impact in signal processing. This thesis focuses on the design of new bases and frames for signal processing and communications. The first contribution of the thesis is the exploration of the use of oversampled filter banks, which represent a possible way to implement frames, to robust communications. Normally, transforms are used as a pure source coding method and a reliable communication is obtained with a channel coder which follows the source coder. In this case, we use frames to achieve efficient source compression and robustness to transmission errors at the same time. In the context of pure source transform coding, we investigate the performance of the wavelet transform and study the dependency of the wavelet coefficients across scales. This analysis leads to the design of a new expansion which provides an efficient representation of piecewise smooth signals. We call footprints the elements of this expansion. The main property of footprints is that they efficiently characterize the singular structures of the signal, which usually carry the main information. We show that algorithms based on footprints outperform wavelet methods in different applications such as denoising, compression and deconvolution. Finally, we study a particular source coding technique called multiple description coding. This technique is used for data transmission over unreliable networks. In multiple description coding, the coder generates several different descriptions of the same signal and the decoder can produce a useful reconstruction of the source with any received subset of these descriptions. We study the problem of multiple description coding of stationary Gaussian sources with memory. First, we compute the multiple description rate distortion region for these sources. Then, we develop an algorithm for the design of optimal critically sampled filter banks for multiple description coding of Gaussian sources.

[1]  Michael T. Orchard,et al.  Multiple description image coding for noisy channels by pairing transform coefficients , 1997, Proceedings of First Signal Processing Society Workshop on Multimedia Signal Processing.

[2]  Robert Bregovic,et al.  Multirate Systems and Filter Banks , 2002 .

[3]  Vinay A. Vaishampayan,et al.  Design of multiple description scalar quantizers , 1993, IEEE Trans. Inf. Theory.

[4]  Ahmad Zandi,et al.  CREW: Compression with Reversible Embedded Wavelets , 1995, Proceedings DCC '95 Data Compression Conference.

[5]  Ronald R. Coifman,et al.  Entropy-based algorithms for best basis selection , 1992, IEEE Trans. Inf. Theory.

[6]  Jonathan Hong,et al.  Discrete Fourier, Hartley, and cosine transforms in signal processing , 1994, Signal Process..

[7]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[8]  Kannan Ramchandran,et al.  Multiple description wavelet based image coding , 2000, IEEE Trans. Image Process..

[9]  Vivek K. Goyal,et al.  Quantized Overcomplete Expansions in IRN: Analysis, Synthesis, and Algorithms , 1998, IEEE Trans. Inf. Theory.

[10]  Vinay A. Vaishampayan,et al.  Asymptotic Analysis of Multiple Description Quantizers , 1998, IEEE Trans. Inf. Theory.

[11]  Ajay Ingle,et al.  DPCM system design for diversity systems with applications to packetized speech , 1995, IEEE Trans. Speech Audio Process..

[12]  Robert M. Gray,et al.  On the asymptotic eigenvalue distribution of Toeplitz matrices , 1972, IEEE Trans. Inf. Theory.

[13]  Vivek K. Goyal,et al.  Multiple description lattice vector quantization: variations and extensions , 2000, Proceedings DCC 2000. Data Compression Conference.

[14]  Minh N. Do,et al.  On the compression of two-dimensional piecewise smooth functions , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[15]  U. Grenander,et al.  Toeplitz Forms And Their Applications , 1958 .

[16]  D. L. Donoho,et al.  Ideal spacial adaptation via wavelet shrinkage , 1994 .

[17]  Zhifeng Zhang,et al.  Adaptive time-frequency decompositions , 1994 .

[18]  Vinay A. Vaishampayan,et al.  Asymptotic performance of multiple description transform codes , 1997, IEEE Trans. Inf. Theory.

[19]  Toby Berger,et al.  Multiple description source coding with no excess marginal rate , 1995, IEEE Trans. Inf. Theory.

[20]  Thomas L. Marzetta,et al.  Systematic design of unitary space-time constellations , 2000, IEEE Trans. Inf. Theory.

[21]  Jaroslaw Domaszewicz,et al.  Design of entropy-constrained multiple-description scalar quantizers , 1994, IEEE Trans. Inf. Theory.

[22]  Abbas El Gamal,et al.  Achievable rates for multiple descriptions , 1982, IEEE Trans. Inf. Theory.

[23]  I. M. Glazman,et al.  Theory of linear operators in Hilbert space , 1961 .

[24]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[25]  Michael Unser,et al.  On the optimality of ideal filters for pyramid and wavelet signal approximation , 1993, IEEE Trans. Signal Process..

[26]  Vivek K. Goyal,et al.  Optimal multiple description transform coding of Gaussian vectors , 1998, Proceedings DCC '98 Data Compression Conference (Cat. No.98TB100225).

[27]  Martin Vetterli,et al.  Data Compression and Harmonic Analysis , 1998, IEEE Trans. Inf. Theory.

[28]  Ingrid Daubechies,et al.  The wavelet transform, time-frequency localization and signal analysis , 1990, IEEE Trans. Inf. Theory.

[29]  Giovanni Poggi,et al.  Compression of multispectral images by three-dimensional SPIHT algorithm , 2000, IEEE Trans. Geosci. Remote. Sens..

[30]  Vivek K. Goyal,et al.  Generalized multiple description coding with correlating transforms , 2001, IEEE Trans. Inf. Theory.

[31]  Michael Unser,et al.  Texture classification and segmentation using wavelet frames , 1995, IEEE Trans. Image Process..

[32]  S. Mallat A wavelet tour of signal processing , 1998 .

[33]  D. Donoho,et al.  Uncertainty principles and signal recovery , 1989 .

[34]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[35]  Martin Vetterli,et al.  Shift-invariant Gibbs-free denoising algorithm based on wavelet transform footprints , 2000, SPIE Optics + Photonics.

[36]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[37]  Martin Vetterli,et al.  Optimal biorthogonal filter banks for multiple description coding , 2000 .

[38]  Robert M. Gray,et al.  Source coding for a simple network , 1974 .

[39]  Vivek K. Goyal,et al.  Quantized oversampled filter banks with erasures , 2001, Proceedings DCC 2001. Data Compression Conference.

[40]  K Ramchandran,et al.  Best wavelet packet bases in a rate-distortion sense , 1993, IEEE Trans. Image Process..

[41]  I. Daubechies,et al.  Wavelets on the Interval and Fast Wavelet Transforms , 1993 .

[42]  Michael T. Orchard,et al.  Redundancy rate-distortion analysis of multiple description coding using pairwise correlating transforms , 1997, Proceedings of International Conference on Image Processing.

[43]  Vinay A. Vaishampayan,et al.  Low-delay communication for Rayleigh fading channels: an application of the multiple description quantizer , 1995, IEEE Trans. Commun..

[44]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[45]  Steven McCanne,et al.  Receiver-driven layered multicast , 1996, SIGCOMM '96.

[46]  Martin Vetterli,et al.  Footprints and edgeprints for image denoising and compression , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[47]  B. Frieden,et al.  Image recovery: Theory and application , 1987, IEEE Journal of Quantum Electronics.

[48]  Martin Vetterli,et al.  Wavelets, approximation, and compression , 2001, IEEE Signal Process. Mag..

[49]  Vivek K. Goyal Transform coding with integer-to-integer transforms , 2000, IEEE Trans. Inf. Theory.

[50]  Jerome M. Shapiro,et al.  Embedded image coding using zerotrees of wavelet coefficients , 1993, IEEE Trans. Signal Process..

[51]  N. J. A. Sloane,et al.  Design of asymmetric multiple description lattice vector quantizers , 2000, Proceedings DCC 2000. Data Compression Conference.

[52]  Minh N. Do,et al.  Rate-distortion optimal tree algorithms for piecewise polynomials , 2002, Proceedings IEEE International Symposium on Information Theory,.

[53]  Martin Vetterli,et al.  Receiver-driven layered multicast , 1996, SIGCOMM 1996.

[54]  L. Ozarow,et al.  On a source-coding problem with two channels and three receivers , 1980, The Bell System Technical Journal.

[55]  Patrick L. Combettes Generalized convex set theoretic image recovery , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[56]  Aggelos K. Katsaggelos,et al.  Spatially adaptive wavelet-based multiscale image restoration , 1996, IEEE Trans. Image Process..

[57]  I. Daubechies,et al.  Wavelet Transforms That Map Integers to Integers , 1998 .

[58]  E. Candès,et al.  Curvelets: A Surprisingly Effective Nonadaptive Representation for Objects with Edges , 2000 .

[59]  Martin Vetterli,et al.  Wavelet footprints: theory, algorithms, and applications , 2003, IEEE Trans. Signal Process..

[60]  Helmut Bölcskei,et al.  Frame-theoretic analysis of oversampled filter banks , 1998, IEEE Trans. Signal Process..

[61]  Michael Elad,et al.  On sparse signal representations , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[62]  Martin Vetterli,et al.  Optimal filter banks for multiple description coding: Analysis and synthesis , 2002, IEEE Trans. Inf. Theory.

[63]  T. Cover,et al.  Rate Distortion Theory , 2001 .

[64]  D. Donoho Nonlinear Solution of Linear Inverse Problems by Wavelet–Vaguelette Decomposition , 1995 .

[65]  Stéphane Mallat,et al.  Singularity detection and processing with wavelets , 1992, IEEE Trans. Inf. Theory.

[66]  Toby Berger,et al.  New results in binary multiple descriptions , 1987, IEEE Trans. Inf. Theory.

[67]  Allen Gersho,et al.  Vector quantization and signal compression , 1991, The Kluwer international series in engineering and computer science.

[68]  Martin Vetterli,et al.  Analysis of optimal filter banks for multiple description coding , 2000, Proceedings DCC 2000. Data Compression Conference.

[69]  R. Duffin,et al.  A class of nonharmonic Fourier series , 1952 .

[70]  Stéphane Mallat,et al.  Characterization of Signals from Multiscale Edges , 2011, IEEE Trans. Pattern Anal. Mach. Intell..

[71]  Vivek K Goyal,et al.  Quantized Frame Expansions with Erasures , 2001 .

[72]  Vivek K. Goyal,et al.  Filter bank frame expansions with erasures , 2002, IEEE Trans. Inf. Theory.

[73]  Martin Vetterli,et al.  Directional wavelet transforms and frames , 2002, Proceedings. International Conference on Image Processing.

[74]  Ram Zamir Gaussian codes and Shannon bounds for multiple descriptions , 1999, IEEE Trans. Inf. Theory.

[75]  O. Nelles,et al.  An Introduction to Optimization , 1996, IEEE Antennas and Propagation Magazine.

[76]  Zhifeng Zhang,et al.  Adaptive Nonlinear Approximations , 1994 .

[77]  Michael T. Orchard,et al.  On the importance of combining wavelet-based nonlinear approximation with coding strategies , 2002, IEEE Trans. Inf. Theory.

[78]  M. Gormish,et al.  Multiscale Sharpening and Smoothing in Besov Spaces with Applications to Image Enhancement , 2001 .

[79]  John J. Benedetto,et al.  Wavelet analysis of spectrogram seizure chirps , 1995, Optics + Photonics.

[80]  N. J. A. Sloane,et al.  Multiple-description vector quantization with lattice codebooks: Design and analysis , 2001, IEEE Trans. Inf. Theory.

[81]  Vivek K. Goyal,et al.  Quantized frame expansions as source-channel codes for erasure channels , 1999, Proceedings DCC'99 Data Compression Conference (Cat. No. PR00096).

[82]  I. Daubechies,et al.  PAINLESS NONORTHOGONAL EXPANSIONS , 1986 .

[83]  Stéphane Mallat,et al.  Image compression with geometrical wavelets , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[84]  Martin Vetterli,et al.  Deconvolution with wavelet footprints for ill-posed inverse problems , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[85]  John J. Benedetto,et al.  Wavelet periodicity detection algorithms , 1998, Optics & Photonics.

[86]  William A. Pearlman,et al.  A new, fast, and efficient image codec based on set partitioning in hierarchical trees , 1996, IEEE Trans. Circuits Syst. Video Technol..

[87]  Antonio Ortega,et al.  Multiresolution broadcast for digital HDTV using joint source-channel coding , 1992, [Conference Record] SUPERCOMM/ICC '92 Discovering a New World of Communications.

[88]  A. Said,et al.  Manuscript Submitted to the Ieee Transactions on Circuits and Systems for Video Technology a New Fast and Eecient Image Codec Based on Set Partitioning in Hierarchical Trees , 2007 .

[89]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[90]  Michael Fleming,et al.  Generalized multiple description vector quantization , 1999, Proceedings DCC'99 Data Compression Conference (Cat. No. PR00096).

[91]  Rudolf Ahlswede,et al.  The rate-distortion region for multiple descriptions without excess rate , 1985, IEEE Trans. Inf. Theory.

[92]  Bixio Rimoldi,et al.  Successive refinement of information: characterization of the achievable rates , 1994, IEEE Trans. Inf. Theory.

[93]  D. Donoho,et al.  Translation-Invariant DeNoising , 1995 .

[94]  Vilius Ivanauskas Conferences , 1979 .

[95]  Vivek K. Goyal,et al.  Multiple description coding: compression meets the network , 2001, IEEE Signal Process. Mag..

[96]  Tamás Linder,et al.  The multiple description rate region for high resolution source coding , 1998, Proceedings DCC '98 Data Compression Conference (Cat. No.98TB100225).

[97]  Stéphane Mallat,et al.  Image deconvolution in mirror wavelet bases , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[98]  Richard G. Baraniuk,et al.  Wavelet-based deconvolution for ill-conditioned systems , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[99]  Arturo R. P. Ragozini,et al.  Fast color transformation by means of hierarchical table lookups , 1998, 9th European Signal Processing Conference (EUSIPCO 1998).

[100]  I. Daubechies,et al.  Factoring wavelet transforms into lifting steps , 1998 .

[101]  Martin Vetterli,et al.  Wavelet transform footprints: catching singularities for compression and denoising , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[102]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[103]  M. Vetterli,et al.  Approximation and compression of piecewise smooth functions , 1999, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[104]  Georgios B. Giannakis,et al.  Principal component filter banks for optimal multiresolution analysis , 1995, IEEE Trans. Signal Process..