Online estimation of MGGD: the Riemannian Averaged Natural Gradient method

Multivariate Generalized Gaussian Distributions (MGGD) are a rich class of multivariate distributions, which have gained importance across many engineering applications (image processing, computer vision, radar and biomedical signal processing). Unfortunately, estimating the parameters of MGGD leads to non-linear matrix equations, whose solution becomes unpractical in high-dimensional problems, or when dealing with very large datasets. To overcome this difficulty, the present paper proposes a new method for online estimation of MGGD parameters, called the Riemannian Averaged Natural Gradient (RANG) method. The RANG method is suitable for application with high-dimensional and large datasets, since it requires modest memory and computational resources. The present paper formulates this new method, and presents some computer simulations, to showcase its performance. It is seen that, while the RANG method makes less exhaustive use of available data, it still achieves identical performance, to classical maximum-likelihood estimation, for sufficiently large datasets.

[1]  Suvrit Sra,et al.  First-order Methods for Geodesically Convex Optimization , 2016, COLT.

[2]  Shun-ichi Amari,et al.  Methods of information geometry , 2000 .

[3]  M. Abramowitz,et al.  Handbook of Mathematical Functions With Formulas, Graphs and Mathematical Tables (National Bureau of Standards Applied Mathematics Series No. 55) , 1965 .

[4]  R. Maronna.  Robust $M$-Estimators of Multivariate Location and Scatter , 1976 .

[5]  Paul Scheunders,et al.  Geodesics on the Manifold of Multivariate Generalized Gaussian Distributions with an Application to Multicomponent Texture Discrimination , 2011, International Journal of Computer Vision.

[6]  Russell M. Mersereau,et al.  Adaptive subband video coding using bivariate generalized Gaussian distribution model , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[7]  Michael I. Jordan,et al.  Averaging Stochastic Gradient Descent on Riemannian Manifolds , 2018, COLT.

[8]  Jacob Scharcanski.  A Wavelet-Based Approach for Analyzing Industrial Stochastic Textures With Applications , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[9]  Tien D. Bui,et al.  Image Denoising Based on Wavelet Shrinkage Using Neighbor and Level Dependency , 2009, Int. J. Wavelets Multiresolution Inf. Process..

[10]  Tülay Adali,et al.  A New Riemannian Averaged Fixed-Point Algorithm for MGGD Parameter Estimation , 2015, IEEE Signal Processing Letters.

[11]  S. Kotz.  Multivariate Distributions at a Cross Road , 1975 .

[12]  Philippe Forster,et al.  Covariance Structure Maximum-Likelihood Estimates in Compound Gaussian Noise: Existence and Algorithm Analysis , 2008, IEEE Transactions on Signal Processing.

[13]  Mohamed-Jalal Fadili,et al.  Multivariate statistical modeling of images with the curvelet transform , 2005, Proceedings of the Eighth International Symposium on Signal Processing and Its Applications, 2005..

[14]  Muralidhar Rangaswamy,et al.  Computer generation of correlated non-Gaussian radar clutter , 1995 .

[15]  Nizar Bouguila,et al.  Bayesian Learning of Generalized Gaussian Mixture Models on Biomedical Images , 2010, ANNPR.

[16]  Ami Wiesel,et al.  Multivariate Generalized Gaussian Distribution: Convexity and Graphical Models , 2013, IEEE Transactions on Signal Processing.

[17]  Peter M. Bentler,et al.  Geodesic Estimation in Elliptical Distributions , 1997 .

[18]  Mukund Desai,et al.  Robust Gaussian and non-Gaussian matched subspace detection , 2003, IEEE Trans. Signal Process..

[19]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[20]  Suvrit Sra,et al.  Conic Geometric Optimization on the Manifold of Positive Definite Matrices , 2013, SIAM J. Optim..

[21]  A. Öztürk,et al.  Non-Gaussian random vector identification using spherically invariant random processes , 1993 .

[22]  Paul Scheunders,et al.  On the Geometry of Multivariate Generalized Gaussian Models , 2012, Journal of Mathematical Imaging and Vision.

[23]  Xavier Pennec,et al.  A Riemannian Framework for Tensor Computing , 2005, International Journal of Computer Vision.

[24]  Paul Scheunders,et al.  Multiscale colour texture retrieval using the geodesic distance between multivariate generalized Gaussian models , 2008, 2008 15th IEEE International Conference on Image Processing.

[25]  Yacine Chitour,et al.  Exact Maximum Likelihood Estimates for SIRV Covariance Matrix: Existence and Algorithm Analysis , 2008, IEEE Transactions on Signal Processing.

[26]  Qionghai Dai,et al.  Image and Video Denoising Using Adaptive Dual-Tree Discrete Wavelet Packets , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[27]  Silvere Bonnabel,et al.  Stochastic Gradient Descent on Riemannian Manifolds , 2011, IEEE Transactions on Automatic Control.

[28]  Salem Said,et al.  Recursive parameter estimation in a Riemannian manifold , 2018 .

[29]  Douglas Kelker,et al.  DISTRIBUTION THEORY OF SPHERICAL DISTRIBUTIONS AND A LOCATION-SCALE PARAMETER GENERALIZATION , 2016 .

[30]  Lorenzo Bruzzone,et al.  Image thresholding based on the EM algorithm and the generalized Gaussian distribution , 2007, Pattern Recognit..

[31]  Jean-Yves Tourneret,et al.  Parameter Estimation For Multivariate Generalized Gaussian Distributions , 2013, IEEE Transactions on Signal Processing.

[32]  Ganapati P. Patil,et al.  Statistical Distributions in Scientific Work , 1981 .

[33]  Levent Tunçel,et al.  Optimization algorithms on matrix manifolds , 2009, Math. Comput..