Filterbank optimization with convex objectives and the optimality of principal component forms

This paper proposes a general framework for the optimization of orthonormal filterbanks (FBs) for given input statistics. This includes as special cases, many previous results on FB optimization for compression. It also solves problems that have not been considered thus far. FB optimization for coding gain maximization (for compression applications) has been well studied before. The optimum FB has been known to satisfy the principal component property, i.e., it minimizes the mean-square error caused by reconstruction after dropping the P weakest (lowest variance) subbands for any P. We point out a much stronger connection between this property and the optimality of the FB. The main result is that a principal component FB (PCFB) is optimum whenever the minimization objective is a concave function of the subband variances produced by the FB. This result has its grounding in majorization and convex function theory and, in particular, explains the optimality of PCFBs for compression. We use the result to show various other optimality properties of PCFBs, especially for noise-suppression applications. Suppose the FB input is a signal corrupted by additive white noise, the desired output is the pure signal, and the subbands of the FB are processed to minimize the output noise. If each subband processor is a zeroth-order Wiener filter for its input, we can show that the expected mean square value of the output noise is a concave function of the subband signal variances. Hence, a PCFB is optimum in the sense of minimizing this mean square error. The above-mentioned concavity of the error and, hence, PCFB optimality, continues to hold even with certain other subband processors such as subband hard thresholds and constant multipliers, although these are not of serious practical interest. We prove that certain extensions of this PCFB optimality result to cases where the input noise is colored, and the FB optimization is over a larger class that includes biorthogonal FBs. We also show that PCFBs do not exist for the classes of DFT and cosine-modulated FBs.

[1]  P. P. Vaidyanathan,et al.  Theory of optimal orthonormal subband coders , 1998, IEEE Trans. Signal Process..

[2]  S. Akkarakaran,et al.  Optimized orthonormal transforms for SNR improvement by subband processing , 1999, 1999 2nd IEEE Workshop on Signal Processing Advances in Wireless Communications (Cat. No.99EX304).

[3]  P. P. Vaidyanathan,et al.  On existence of FIR principal component filter banks , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[4]  Mihai Anitescu,et al.  Theory of rate-distortion-optimal, constrained filterbanks-application to IIR and FIR biorthogonal designs , 2000, IEEE Trans. Signal Process..

[5]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[6]  Irving Kalet,et al.  The multitone channel , 1989, IEEE Trans. Commun..

[7]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[8]  丸山 徹.  Convex Analysisの二,三の進展について , 1977 .

[9]  P. P. Vaidyanathan,et al.  Results on principal component filter banks: Colored noise suppression and existence issues , 2001, IEEE Trans. Inf. Theory.

[10]  P. P. Vaidyanathan,et al.  Results on optimal biorthogonal filter banks , 1998 .

[11]  P. Schultheiss,et al.  Block Quantization of Correlated Gaussian Random Variables , 1963 .

[12]  P. P. Vaidyanathan,et al.  OPTIMALITY OF ORTHONORMAL TRANSFORMS FOR SUBBAND CODING , 1998 .

[13]  Pierre Moulin,et al.  Theory and design of signal-adapted FIR paraunitary filter banks , 1998, IEEE Trans. Signal Process..

[14]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[15]  I. Olkin,et al.  Inequalities: Theory of Majorization and Its Applications , 1980 .

[16]  Michelle Effros,et al.  On the rate-distortion performance and computational efficiency of the Karhunen-Loeve transform for lossy data compression , 2002, IEEE Trans. Image Process..

[17]  Georgios B. Giannakis,et al.  Principal component filter banks for optimal multiresolution analysis , 1995, IEEE Trans. Signal Process..

[18]  P. P. Vaidyanathan,et al.  Optimality of principal component filter banks for discrete multitone communication systems , 2000, 2000 IEEE International Symposium on Circuits and Systems. Emerging Technologies for the 21st Century. Proceedings (IEEE Cat No.00CH36353).

[19]  P. P. Vaidyanathan,et al.  Nonuniform principal component filter banks: definitions, existence, and optimality , 2000, SPIE Optics + Photonics.

[20]  P. P. Vaidyanathan,et al.  On optimization of filter banks with denoising applications , 1999, ISCAS'99. Proceedings of the 1999 IEEE International Symposium on Circuits and Systems VLSI (Cat. No.99CH36349).

[21]  P. P. Vaidyanathan,et al.  Principal component filter banks: existence issues, and application to modulated filter banks , 2000, 2000 IEEE International Symposium on Circuits and Systems. Emerging Technologies for the 21st Century. Proceedings (IEEE Cat No.00CH36353).