Kullback–Leibler Divergence Between Multivariate Generalized Gaussian Distributions

The Kullback–Leibler divergence (KLD) between two multivariate generalized Gaussian distributions (MGGDs) is a fundamental tool in many signal and image processing applications. Until now, the KLD of MGGDs has no known explicit form, and it is in practice either estimated using expensive Monte-Carlo stochastic integration or approximated. The main contribution of this letter is to present a closed-form expression of the KLD between two zero-mean MGGDs. Depending on the Lauricella series, a simple way of calculating numerically the KLD is exposed. Finally, we show that the approximation of the KLD by Monte-Carlo sampling converges to its theoretical value when the number of samples goes to the infinity.

[1]  G. Lauricella,et al.  Sulle funzioni ipergeometriche a piu variabili , 1893 .

[2]  Tien D. Bui,et al.  Multivariate statistical modeling for image denoising using wavelet transforms , 2005, Signal Process. Image Commun..

[3]  Harold Exton,et al.  Multiple hypergeometric functions and applications , 1979 .

[4]  M. A. Gómez–Villegas,et al.  A MATRIX VARIATE GENERALIZATION OF THE POWER EXPONENTIAL FAMILY OF DISTRIBUTIONS , 2002 .

[5]  Tien D. Bui,et al.  Image Denoising Based on Wavelet Shrinkage Using Neighbor and Level Dependency , 2009, Int. J. Wavelets Multiresolution Inf. Process..

[6]  Frank Nielsen,et al.  Guaranteed Bounds on the Kullback–Leibler Divergence of Univariate Mixtures , 2016, IEEE Signal Processing Letters.

[7]  Paul Scheunders,et al.  Wavelet-based colour texture retrieval using the kullback-leibler divergence between bivariate generalized Gaussian models , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[8]  Levent Sendur,et al.  A bivariate shrinkage function for wavelet-based denoising , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[9]  Paul Scheunders,et al.  Geodesics on the Manifold of Multivariate Generalized Gaussian Distributions with an Application to Multicomponent Texture Discrimination , 2011, International Journal of Computer Vision.

[10]  Sarah Rothstein.  Jacobians Of Matrix Transformations And Functions Of Matrix Argument , 2016 .

[11]  Shiyong Cui,et al.  Statistical Wavelet Subband Modeling for Multi-Temporal SAR Change Detection , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[12]  Amel Benazza-Benyahia,et al.  Close Approximation of Kullback–Leibler Divergence for Sparse Source Retrieval , 2019, IEEE Signal Processing Letters.

[13]  A. Hattori,et al.  On the Euler integral representations of hypergeometric functions in several variables , 1974 .

[14]  Jean-Yves Tourneret,et al.  Parameter Estimation For Multivariate Generalized Gaussian Distributions , 2013, IEEE Transactions on Signal Processing.

[15]  Yannick Berthoumieu,et al.  Multivariate texture retrieval using the Kullback-Leibler divergence between bivariate generalized Gamma times an Uniform distribution , 2012, 2012 19th IEEE International Conference on Image Processing.

[16]  Shiyong Cui,et al.  Comparison of Kullback-Leibler divergence approximation methods between Gaussian mixture models for satellite image retrieval , 2015, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[17]  Minh N. Do,et al.  Wavelet-based texture retrieval using generalized Gaussian density and Kullback-Leibler distance , 2002, IEEE Trans. Image Process..