Nonasymptotic Gaussian Approximation for Linear Systems with Stable Noise [Preliminary Version]

The results of a series of theoretical studies are reported, examining the convergence rate for different approximate representations of $\alpha$-stable distributions. Although they play a key role in modelling time-series with jumps and discontinuities, the use of $\alpha$-stable distributions in inference often leads to analytically intractable problems. The infinite series representation employed in this work transforms an intractable, infinite-dimensional inference problem into a finite-dimensional (conditionally Gaussian) parametric problem. The main gist of our approach is the approximation of the tail of this series by a Gaussian random variable. Standard statistical techniques, such as Expectation-Maximization, Markov chain Monte Carlo, and Particle Filtering, can then be readily applied. In addition to the asymptotic normality of the tail of this series, here we establish explicit, nonasymptotic bounds on the approximation error. Their proofs follow classical Fourier-analytic arguments, typically employing Ess\'{e}en's smoothing lemma. Specifically, we consider the distance between the distributions of: $(i)$~the tail of the series and an appropriate Gaussian; $(ii)$~the full series and the truncated series; and $(iii)$~the full series and the truncated series with an added Gaussian term. In all three cases, sharp bounds are established, and the theoretical results are compared with the actual distances (computed numerically) in specific examples of symmetric $\alpha$-stable distributions. This analysis facilitates the selection of appropriate truncations in practice and offers theoretical guarantees for the accuracy of the resulting estimates. One of the main conclusions obtained is that the use of a truncated series together with an approximately Gaussian error term has superior statistical properties and is probably preferable for inference tasks.

[1]  Simon J. Godsill,et al.  Simulated convergence rates with application to an intractable α-stable inference problem , 2017, 2017 IEEE 7th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[2]  Simon J. Godsill,et al.  Convergence results for tractable inference in α-stable stochastic processes , 2017, 2017 22nd International Conference on Digital Signal Processing (DSP).

[3]  Simon J. Godsill,et al.  Approximate simulation of linear continuous time models driven by asymmetric stable Lévy processes , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  Brian M. Sadler,et al.  Robust compressive sensing of sparse signals: a review , 2016, EURASIP J. Adv. Signal Process..

[5]  Simon J. Godsill,et al.  Fully Bayesian inference for α-stable distributions using a Poisson series representation , 2015, Digit. Signal Process..

[6]  Simon J. Godsill,et al.  Inference for models with asymmetric α -stable noise processes , 2015 .

[7]  Miles E. Lopes Compressed Sensing without Sparsity Assumptions , 2015, ArXiv.

[8]  Michael Unser,et al.  An Introduction to Sparse Stochastic Processes , 2014 .

[9]  Simon J. Godsill,et al.  A poisson series approach to Bayesian Monte Carlo inference for skewed alpha-stable distributions , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[10]  Michael Unser,et al.  Sparsity and Infinite Divisibility , 2014, IEEE Transactions on Information Theory.

[11]  Edward Neuman,et al.  Inequalities and Bounds for the Incomplete Gamma Function , 2013 .

[12]  Andreas Winkelbauer,et al.  Moments and Absolute Moments of the Normal Distribution , 2012, ArXiv.

[13]  Simon J. Godsill,et al.  Linear gaussian computations for near-exact Bayesian Monte Carlo inference in skewed alpha-stable time series models , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[14]  Michael Unser,et al.  A unified formulation of Gaussian vs. sparse stochastic processes - Part II: Discrete-domain theory , 2011, ArXiv.

[15]  Michael Unser,et al.  A unified formulation of Gaussian vs. sparse stochastic processes - Part I: Continuous-domain theory , 2011, ArXiv.

[16]  Simon J. Godsill,et al.  Enhanced Poisson sum representation for alpha-stable processes , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[17]  Alin Achim,et al.  Compressive sensing for ultrasound RF echoes using a-Stable Distributions , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[18]  Xiaohui Chen,et al.  Asymptotic Analysis of Robust LASSOs in the Presence of Noise With Large Variance , 2010, IEEE Transactions on Information Theory.

[19]  Florin Ciucu,et al.  Delay Bounds in Communication Networks With Heavy-Tailed and Self-Similar Traffic , 2009, IEEE Transactions on Information Theory.

[20]  Peter J. Brockwell,et al.  Existence and uniqueness of stationary Lvy-driven CARMA processes , 2009 .

[21]  Simon J. Godsill,et al.  An Overview of Existing Methods and Recent Advances in Sequential Monte Carlo , 2007, Proceedings of the IEEE.

[22]  Marco J. Lombardi Bayesian inference for α-stable distributions: A random walk MCMC approach , 2007 .

[23]  George Michailidis,et al.  Estimating Heavy-Tail Exponents Through Max Self–Similarity , 2006, IEEE Transactions on Information Theory.

[24]  Josiane Zerubia,et al.  SAR image filtering based on the heavy-tailed Rayleigh model , 2006, IEEE Transactions on Image Processing.

[25]  Marios M. Polycarpou,et al.  Adaptive and Learning Systems for Signal Processing, Communications, and Control , 2006 .

[26]  Kiseon Kim,et al.  Robust minimax detection of a weak signal in noise with a bounded variance and density value at the center of symmetry , 2006, IEEE Transactions on Information Theory.

[27]  Thomas B. Schön,et al.  Marginalized particle filters for mixed linear/nonlinear state-space models , 2005, IEEE Transactions on Signal Processing.

[28]  Peter J. Brockwell,et al.  Representations of continuous-time ARMA processes , 2004, Journal of Applied Probability.

[29]  M. Parlange,et al.  Statistics of extremes in hydrology , 2002 .

[30]  Ercan E. Kuruoglu,et al.  Density parameter estimation of skewed α-stable distributions , 2001, IEEE Trans. Signal Process..

[31]  Vidmantas Bentkus,et al.  Lévy–LePage Series Representation of Stable Vectors: Convergence in Variation , 2001 .

[32]  Alin Achim,et al.  Novel Bayesian multiscale method for speckle removal in medical ultrasound images , 2001, IEEE Transactions on Medical Imaging.

[33]  S. Asmussen,et al.  Approximations of small jumps of Lévy processes with a view towards simulation , 2001, Journal of Applied Probability.

[34]  Simon J. Godsill,et al.  On sequential Monte Carlo sampling methods for Bayesian filtering , 2000, Stat. Comput..

[35]  Simon J. Godsill,et al.  Inference in symmetric alpha-stable noise using MCMC and the slice sampler , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[36]  S. Godsill MCMC and EM-based methods for inference in heavy-tailed processes with /spl alpha/-stable innovations , 1999, Proceedings of the IEEE Signal Processing Workshop on Higher-Order Statistics. SPW-HOS '99.

[37]  S. Godsill,et al.  Bayesian inference for time series with heavy-tailed symmetric α-stable noise processes , 1999 .

[38]  E. Tsionas Monte Carlo inference in econometric models with symmetric stable disturbances , 1999 .

[39]  Richard A. Davis,et al.  Introduction to time series and forecasting , 1998 .

[40]  Marvin M. Kilgo,et al.  Introduction to Time Series and Forecasting , 1998 .

[41]  Zuqiang Qiou,et al.  Bayesian Inference for Time Series with Stable Innovations , 1998 .

[42]  Friedrich Götze,et al.  Bounds for the accuracy of Poissonian approximations of stable laws , 1996 .

[43]  M. Ledoux,et al.  A rate of convergence in the poissonian representation of stable distributions , 1996 .

[44]  W. Linde STABLE NON‐GAUSSIAN RANDOM PROCESSES: STOCHASTIC MODELS WITH INFINITE VARIANCE , 1996 .

[45]  R. Weron Correction to: "On the Chambers–Mallows–Stuck Method for Simulating Skewed Stable Random Variables" , 1996 .

[46]  D. Buckle Bayesian Inference for Stable Distributions , 1995 .

[47]  M. Taqqu,et al.  Stable Non-Gaussian Random Processes : Stochastic Models with Infinite Variance , 1995 .

[48]  M. Tanner Tools for statistical inference: methods for the exploration of posterior distributions and likeliho , 1994 .

[49]  R. Fildes Forecasting structural time series models and the kalman filter: Andrew Harvey, 1989, (Cambridge University Press), 554 pp., ISBN 0-521-32196-4 , 1992 .

[50]  R. Katz,et al.  Extreme events in a changing climate: Variability is more important than averages , 1992 .

[51]  John B. Thomas,et al.  Asymptotically robust detection and estimation for very heavy-tailed noise , 1991, IEEE Trans. Inf. Theory.

[52]  B. Øksendal Stochastic differential equations : an introduction with applications , 1987 .

[53]  Joel Zinn,et al.  Convergence to a Stable Distribution Via Order Statistics , 1981 .

[54]  Ioannis A. Koutrouvelis,et al.  Regression-Type Estimation of the Parameters of Stable Laws , 1980 .

[55]  Raoul LePage,et al.  Appendix Multidimensional infinitely divisible variables and processes. Part I: Stable case , 1980 .

[56]  C. Mallows,et al.  A Method for Simulating Stable Random Variables , 1976 .

[57]  D. Owen Handbook of Mathematical Functions with Formulas , 1965 .

[58]  B. Mandelbrot New Methods in Statistical Economics , 1963, Journal of Political Economy.

[59]  W. Feller An Introduction to Probability Theory and Its Applications , 1959 .

[60]  W. Feller,et al.  An Introduction to Probability Theory and its Applications , 1958 .

[61]  K. Chung,et al.  Limit Distributions for Sums of Independent Random Variables , 1955 .

[62]  S. Godsill,et al.  Sharp Gaussian Approximation Bounds for Linear Systems with α-stable Noise , 2018 .

[63]  Marina Riabiz,et al.  A central limit theorem with application to inference in α-stable regression models , 2016 .

[64]  S. Godsill,et al.  Pseudo-Marginal MCMC for Parameter Estimation in α-Stable Distributions , 2015 .

[65]  Robert Stelzer,et al.  Lévy-driven CARMA Processes , 2015 .

[66]  Stoyan V. Stoyanov,et al.  Probability Distances and Probability Metrics: Definitions , 2013 .

[67]  A. Doucet,et al.  A Tutorial on Particle Filtering and Smoothing: Fifteen years later , 2008 .

[68]  Marco J. Lombardi,et al.  On-line Bayesian Estimation of Signals in Symmetric α-Stable Noise , 2004 .

[69]  R. E. Kalman,et al.  A New Approach to Linear Filtering and Prediction Problems , 2002 .

[70]  O. Barndorff-Nielsen,et al.  Lévy processes : theory and applications , 2001 .

[71]  Piotr Kokoszka,et al.  Computer investigation of the Rate of Convergence of Lepage Type Series to α-Stable Random Variables , 1992 .

[72]  J. McCulloch,et al.  Simple consistent estimators of stable distribution parameters , 1986 .

[73]  Raoul LePage,et al.  Multidimensional infinitely divisidle variables and processes Part II , 1981 .

[74]  A. G. Greenhill,et al.  Handbook of Mathematical Functions with Formulas, Graphs, , 1971 .

[75]  E. Fama The Behavior of Stock-Market Prices , 1965 .