On the variance of a centered random value roundoff error

We derive two expressions for roundoff error variance, one for a rounded off random value with a zero mean and a given variance under uniform distribution and another for such a value under a normal. Also, an expression for truncation error variance for values under uniform distribution is obtained. An application of the expressions to analysis of processing essentially quantized data by a nonrecursive smoothing filter is shown. Also their applications to quantization error (quantization noise) analysis of general linear processing of quantized signals under uniform and normal distributions and to quantization error analysis of essentially quantized discrete transforms like DFT (discrete Fourier transform), DCT (discrete cosine transform), DWT (discrete Walsh transforms), wavelet transforms, and so on, to image, sound (audio), video and to general signal processing in many cases can be considered as useful. The effect of accuracy of using these expressions is the more, the more used quantization level and the less maximal signal amplitudes. We find the means and variances of the roundoff errors for a centered random variable in a given range.We derive exact expressions for the variance in closed analitical form.We obtain the variance expressions for the cases of uniformly and normally distributed random values.

[1]  Tom Burr,et al.  Rounding error effects in the presence of underlying measurement error , 2012, Accreditation and Quality Assurance.

[2]  Benoît Champagne,et al.  On the steady-state mean squared error of the fixed-point LMS algorithm , 2007, Signal Process..

[3]  Tao Wu,et al.  On normal realizations of digital filters with minimum roundoff noise gain , 2009, Signal Process..

[4]  P. J. B. Koeck Quantization errors in averaged digitized data , 2001, Signal Process..

[5]  Charles M. Rader,et al.  Effects of quantization noise in digital filters , 1966, AFIPS '66 (Spring).

[6]  Hong Gu,et al.  Fixed-point error analysis and an efficient array processor design of two-dimensional sliding DFT , 1999, Signal Process..

[7]  R. Stevenson,et al.  DCT quantization noise in compressed images , 2005 .

[8]  John Vanderkooy,et al.  Quantization and Dither: A Theoretical Survey , 1992 .

[9]  Shu Hung Leung,et al.  On the statistics of fixed-point roundoff error , 1985, IEEE Trans. Acoust. Speech Signal Process..

[10]  Uwe Ullrich,et al.  Roundoff noise and dynamic range of wave digital filters , 1979 .

[11]  Hanoch Ur,et al.  Quantization noise analysis and error feedback implementation in frequency sampling FIR filters , 1993, Signal Process..

[12]  Bernard Widrow,et al.  Quantization Noise: Roundoff Error in Digital Computation, Signal Processing, Control, and Communications , 2008 .

[13]  A. Turing ROUNDING-OFF ERRORS IN MATRIX PROCESSES , 1948 .

[14]  James Hardy Wilkinson,et al.  Rounding errors in algebraic processes , 1964, IFIP Congress.

[15]  H. W. Schüssler On the influence of noiselike errors in digital systems , 1983 .