Analysis of Finite Word-Length Effects in Fixed-Point Systems

Systems based on fixed-point arithmetic, when carefully designed, seem to behave as their infinite precision analogues. Most often, however, this is only a macroscopic impression: finite word-lengths inevitably approximate the reference behavior introducing quantization errors, and confine the macroscopic correspondence to a restricted range of input values. Understanding these differences is crucial to design optimized fixed-point implementations that will behave “as expected” upon deployment. Thus, in this chapter, we survey the main approaches proposed in literature to model the impact of finite precision in fixed-point systems. In particular, we focus on the rounding errors introduced after reducing the number of least-significant bits in signals and coefficients during the so-called quantization process.

[1]  Romuald Rocher,et al.  Fixed-Point Configurable Hardware Components , 2006, EURASIP J. Embed. Syst..

[2]  Tomofumi Yuki,et al.  Toward scalable source level accuracy analysis for floating-point to fixed-point conversion , 2014, 2014 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[3]  Octavio Nieto-Taladriz,et al.  Fast and accurate computation of the roundoff noise of linear time-invariant systems , 2008, IET Circuits Devices Syst..

[4]  A. Michel,et al.  Stability analysis of fixed- point digital filters using computer generated Lyapunov functions- Part I: Direct form and coupled form filters , 1985 .

[5]  Sparsh Mittal,et al.  A Survey of Techniques for Approximate Computing , 2016, ACM Comput. Surv..

[6]  Gang Li,et al.  Roundoff noise analysis of two efficient digital filter structures , 2006, IEEE Transactions on Signal Processing.

[7]  Rob A. Rutenbar,et al.  Floating-point error analysis based on affine arithmetic , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[8]  David Novo,et al.  Fast performance evaluation of fixed-point systems with un-smooth operators , 2010, 2010 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[9]  A. Sripad,et al.  A necessary and sufficient condition for quantization errors to be uniform and white , 1977 .

[10]  T. Bose,et al.  Overflow oscillations in state-space digital filters , 1991 .

[11]  Joos Vandewalle,et al.  Simulated‐annealing‐based optimization of coefficient and data word‐lengths in digital filters , 1988 .

[12]  Wayne Luk,et al.  Instrumented Multi-Stage Word-Length Optimization , 2007, 2007 International Conference on Field-Programmable Technology.

[13]  Fabien Rico,et al.  The Fixed CADNA library , 2003 .

[14]  Michael G. Taylor,et al.  Overflow oscillations in digital filters , 1969 .

[15]  Romuald Rocher,et al.  Analytical accuracy evaluation of fixed-point systems , 2007, 2007 15th European Signal Processing Conference.

[16]  P. Perona,et al.  Bit-width optimization for configurable DSP's by multi-interval analysis , 2000, Conference Record of the Thirty-Fourth Asilomar Conference on Signals, Systems and Computers (Cat. No.00CH37154).

[17]  Takao Hinamoto,et al.  L/sub 2/-sensitivity minimization of one- and two-dimensional state-space digital filters subject to L/sub 2/-scaling constraints , 2006, IEEE Transactions on Signal Processing.

[18]  Masayuki Kawamata,et al.  Synthesis of low coefficient sensitivity digital filters using genetic programming , 1999, ISCAS'99. Proceedings of the 1999 IEEE International Symposium on Circuits and Systems VLSI (Cat. No.99CH36349).

[19]  W. R. Bennett,et al.  Spectra of quantized signals , 1948, Bell Syst. Tech. J..

[20]  Wayne Luk,et al.  Floating-point bitwidth analysis via automatic differentiation , 2002, 2002 IEEE International Conference on Field-Programmable Technology, 2002. (FPT). Proceedings..

[21]  Rob A. Rutenbar,et al.  Fast, Accurate Static Analysis for Fixed-Point Finite-Precision Effects in DSP Designs , 2003, ICCAD 2003.

[22]  Peter H. Bauer,et al.  A computer-aided test for the absence of limit cycles in fixed-point digital filters , 1991, IEEE Trans. Signal Process..

[23]  S. Parker,et al.  Correlated noise due to roundoff in fixed point digital filters , 1976 .

[24]  Fernando Cruz-Roldán,et al.  Tighter limit cycle bounds for digital filters , 2006, IEEE Signal Processing Letters.

[25]  A. W. M. van den Enden,et al.  Discrete Time Signal Processing , 1989 .

[26]  Philippe Chevrel,et al.  Sensitivity-Based Pole and Input-Output Errors of Linear Filters as Indicators of the Implementation Deterioration in Fixed-Point Context , 2011, EURASIP J. Adv. Signal Process..

[27]  T. Claasen,et al.  Effects of quantization and overflow in recursive digital filters , 1976 .

[28]  Gian Antonio Mian,et al.  Stability analysis of second-order direct-form digital filters with two roundoff quantizers , 1986 .

[29]  George A. Constantinides,et al.  Word-length optimization for differentiable nonlinear systems , 2006, TODE.

[30]  Takashi Horiyama,et al.  Minimization of fractional wordlength on fixed-point conversion for high-level synthesis , 2004, ASP-DAC 2004: Asia and South Pacific Design Automation Conference 2004 (IEEE Cat. No.04EX753).

[31]  Nicola Nicolici,et al.  Computational Vector-Magnitude-Based Range Determination for Scientific Abstract Data Types , 2011, IEEE Transactions on Computers.

[32]  Florent de Dinechin,et al.  Assisted verification of elementary functions using Gappa , 2006, SAC.

[33]  Gabriel Caffarena,et al.  SQNR Estimation of Fixed-Point DSP Algorithms , 2010, EURASIP J. Adv. Signal Process..

[34]  Alice C. Parker,et al.  Accuracy sensitive word-length selection for algorithm optimization , 1998, Proceedings International Conference on Computer Design. VLSI in Computers and Processors (Cat. No.98CB36273).

[35]  Wonyong Sung,et al.  Optimization of Number Representations , 2018, Handbook of Signal Processing Systems.

[36]  A. Fettweis,et al.  Some principles of designing digital filters imitating classical filter structures , 1971, IEEE Transactions on Circuit Theory.

[37]  Romuald Rocher,et al.  Automatic SQNR determination in non-linear and non-recursive fixed-point systems , 2004, 2004 12th European Signal Processing Conference.

[38]  B. Widrow,et al.  Statistical theory of quantization , 1996 .

[39]  Romuald Rocher,et al.  Analytical Fixed-Point Accuracy Evaluation in Linear Time-Invariant Systems , 2008, IEEE Transactions on Circuits and Systems I: Regular Papers.

[40]  Heinrich Meyr,et al.  Integer code generation for the TI TMS320C62X , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[41]  M. Kawamata,et al.  Heuristic synthesis of low coefficient sensitivity second-order digital filters using genetic programming , 2001 .

[42]  W. Mecklenbrauker,et al.  Second-order digital filter with only one magnitude-truncation quantiser and having practically no limit cycles , 1973 .

[43]  Mark Zwolinski,et al.  Symbolic noise analysis approach to computational hardware optimization , 2008, 2008 45th ACM/IEEE Design Automation Conference.

[44]  Octavio Nieto-Taladriz,et al.  Fast characterization of the noise bounds derived from coefficient and signal quantization , 2003, Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS '03..

[45]  Romuald Rocher,et al.  Noise probability density function in fixed-point systems based on smooth operators , 2012, Proceedings of the 2012 Conference on Design and Architectures for Signal and Image Processing.

[46]  Wayne Luk,et al.  Ieee Transactions on Computer-aided Design of Integrated Circuits and Systems Accuracy Guaranteed Bit-width Optimization Abstract— We Present Minibit, an Automated Static Approach for Optimizing Bit-widths of Fixed-point Feedforward Designs with Guaranteed Accuracy. Methods to Minimize Both the In- , 2022 .

[47]  Gerhard J. Woeginger,et al.  The complexity of multiple wordlength assignment , 2002, Appl. Math. Lett..

[48]  Wayne Luk,et al.  MiniBit: bit-width optimization via affine arithmetic , 2005, Proceedings. 42nd Design Automation Conference, 2005..

[49]  Clifford T. Mullis,et al.  Digital filter realizations without overflow oscillations , 1978, ICASSP.

[50]  Robert W. Brodersen,et al.  A perturbation theory on statistical quantization effects in fixed-point DSP with non-stationary inputs , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[51]  Nicola Nicolici,et al.  Bit-Width Allocation for Hardware Accelerators for Scientific Computing Using SAT-Modulo Theory , 2010, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[52]  Octavio Nieto-Taladriz,et al.  Improved Interval-Based Characterization of Fixed-Point LTI Systems With Feedback Loops , 2007, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[53]  Fausto Pedro García Márquez,et al.  Digital Filters And Signal Processing , 2014 .

[54]  Rob A. Rutenbar,et al.  Lightweight Floating-Point Arithmetic: Case Study of Inverse Discrete Cosine Transform , 2002, EURASIP J. Adv. Signal Process..

[55]  L. Jackson Roundoff noise bounds derived from coefficient sensitivities for digital filters , 1976 .

[56]  O. Sarbishei,et al.  On the Fixed-Point Accuracy Analysis and Optimization of Polynomial Specifications , 2013, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[57]  Stephen A. Dyer,et al.  Digital signal processing , 2018, 8th International Multitopic Conference, 2004. Proceedings of INMIC 2004..

[58]  David Novo,et al.  Quantization mode opportunities in fixed-point system design , 2010, 2010 18th European Signal Processing Conference.

[59]  Seehyun Kim,et al.  Fixed-point optimization utility for C and C++ based digital signal processing programs , 1998 .

[60]  Jianwen Zhu,et al.  Dynamic range estimation for nonlinear systems , 2004, IEEE/ACM International Conference on Computer Aided Design, 2004. ICCAD-2004..

[61]  Hans-Jurgen Butterweck,et al.  New second-order digital filter sections without limit cycles , 1984 .

[62]  Guillaume Melquiond,et al.  Certification of bounds on expressions involving rounded operators , 2007, TOMS.

[63]  Wayne Luk,et al.  Wordlength optimization for linear digital signal processing , 2003, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[64]  Nicola Nicolici,et al.  Automated Range and Precision Bit-Width Allocation for Iterative Computations , 2011, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[65]  Jianwen Zhu,et al.  An analytical approach for dynamic range estimation , 2004, Proceedings. 41st Design Automation Conference, 2004..

[66]  Karthick Parashar,et al.  Leveraging power spectral density for scalable system-level accuracy evaluation , 2016, 2016 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[67]  O. Sarbishei,et al.  Analytical Optimization of Bit-Widths in Fixed-Point LTI Systems , 2012, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[68]  Lothar Thiele,et al.  Design of sensitivity and round-off noise optimal state-space discrete systems , 1984 .

[69]  A. Michel,et al.  Stability analysis of fixed- point digital filters using computer generated Lyapunov functions- Part II: Wave digital filters and lattice digital filters , 1985 .

[70]  Olivier Sentieys,et al.  A methodology for evaluating the precision of fixed-point systems , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[71]  A. Fettweis Wave digital filters: Theory and practice , 1986, Proceedings of the IEEE.

[72]  Octavio Nieto-Taladriz,et al.  Analysis of limit cycles by means of affine arithmetic computer-aided tests , 2004, 2004 12th European Signal Processing Conference.

[73]  Sanjit K. Mitra Digital Signal Processing Laboratory Using MATLAB , 1999 .

[74]  Christian Haubelt,et al.  SystemCoDesigner—an automatic ESL synthesis approach by design space exploration and behavioral synthesis for streaming applications , 2009, TODE.

[75]  Takao Hinamoto,et al.  Minimization of L/sub 2/-sensitivity for state-space digital filters subject to L/sub 2/-dynamic-range scaling constraints , 2005, IEEE Transactions on Circuits and Systems II: Express Briefs.

[76]  Yang Liu,et al.  Collaborative Security , 2015, ACM Comput. Surv..

[77]  Wayne Luk,et al.  Unifying bit-width optimisation for fixed-point and floating-point designs , 2004, 12th Annual IEEE Symposium on Field-Programmable Custom Computing Machines.

[78]  M. Gevers,et al.  Performance analysis of a new structure for digital filter implementation , 2000 .

[79]  Wonyong Sung,et al.  AUTOSCALER for C: an optimizing floating-point to integer C program converter for fixed-point digital signal processors , 2000 .

[80]  David Novo,et al.  Energy efficient MIMO processing: A case study of opportunistic run-time approximations , 2014, 2014 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[81]  Yvon Savaria,et al.  A Metric for Automatic Word-Length Determination of Hardware Datapaths , 2006, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[82]  Daniel Ménard,et al.  Automatic evaluation of the accuracy of fixed-point algorithms , 2002, Proceedings 2002 Design, Automation and Test in Europe Conference and Exhibition.

[83]  George A. Constantinides,et al.  A Scalable Precision Analysis Framework , 2013, IEEE Transactions on Multimedia.

[84]  M. Buttner Elimination of limit cycles in digital filters with very low increase in the quantization noise , 1977 .

[85]  Shu Hung Leung,et al.  On the statistics of fixed-point roundoff error , 1985, IEEE Trans. Acoust. Speech Signal Process..

[86]  Z. Zhao,et al.  On the generalized DFIIt structure and its state-space realization in digital filter implementation , 2004, IEEE Transactions on Circuits and Systems I: Regular Papers.

[87]  P. P. Vaidyanathan,et al.  An improved sufficient condition for absence of limit cycles in digital filters , 1987 .

[88]  Antje Strauss Computer Based Exercises For Signal Processing Using Matlab , 2016 .

[90]  Takao Hinamoto,et al.  Analysis and minimization of L/sub 2/-sensitivity for linear systems and two-dimensional state-space filters using general controllability and observability Gramians , 2002 .

[91]  Karthick Parashar,et al.  Analytical approach for analyzing quantization noise effects on decision operators , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[92]  Edward A. Lee,et al.  Taming heterogeneity - the Ptolemy approach , 2003, Proc. IEEE.

[93]  Karthick Parashar,et al.  Accelerated Performance Evaluation of Fixed-Point Systems With Un-Smooth Operations , 2014, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[94]  Daniel Ménard,et al.  Stochastic modeling for floating-point to fixed-point conversion , 2011, 2011 IEEE Workshop on Signal Processing Systems (SiPS).

[95]  L. Esteban,et al.  Quantization analysis of the infrared interferometer of the TJ-II for its optimized FPGA-based implementation , 2012, 2012 18th IEEE-NPSS Real Time Conference.

[96]  L. Esteban,et al.  Quantization Analysis of the Infrared Interferometer of the TJ-II Stellarator for its Optimized FPGA-Based Implementation , 2012, IEEE Transactions on Nuclear Science.

[97]  Scott Hauck,et al.  Precis: a usercentric word-length optimization tool , 2005, IEEE Design & Test of Computers.

[98]  Robert W. Brodersen,et al.  Floating-point to fixed-point conversion with decision errors due to quantization , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[99]  Tamal Bose,et al.  Stability of digital filters implemented with two's complement truncation quantization , 1992, IEEE Trans. Signal Process..

[100]  L. Jackson Limit cycles in state-space structures for digital filters , 1979 .

[101]  Keshab K. Parhi,et al.  VLSI digital signal processing systems , 1999 .

[102]  Thibault Hilaire Low-Parametric-Sensitivity Realizations With Relaxed $L_{2}$-Dynamic-Range-Scaling Constraints , 2009, IEEE Transactions on Circuits and Systems II: Express Briefs.

[103]  A Davis Timothy,et al.  アルゴリズム907: 回路シミュレーション問題のための直接疎ソルバ,KLU , 2011 .

[104]  Dong-U Lee,et al.  A Bit-Width Optimization Methodology for Polynomial-Based Function Evaluation , 2007, IEEE Transactions on Computers.

[105]  Heinrich Meyr,et al.  Fast bit-true simulation , 2001, Proceedings of the 38th Design Automation Conference (IEEE Cat. No.01CH37232).

[106]  Juan Antonio,et al.  Evaluación de los efectos de cuantificación en las estructuras de filtros digitales mediante técnicas de simulación basadas en extensiones de intervalos. , 2011 .

[107]  N. Wong,et al.  A generalized direct-form delta operator-based IIR filter with minimum noise gain and sensitivity , 2001 .

[108]  G. Alefeld,et al.  Introduction to Interval Computation , 1983 .

[109]  Wayne Luk,et al.  Roundoff-noise shaping in filter design , 2000, 2000 IEEE International Symposium on Circuits and Systems. Emerging Technologies for the 21st Century. Proceedings (IEEE Cat No.00CH36353).

[110]  Katarzyna Radecka,et al.  Optimization of Imprecise Circuits Represented by Taylor Series and Real-Valued Polynomials , 2010, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[111]  M. Kawamata,et al.  Evolutionary synthesis of digital filter structures using genetic programming , 2003, IEEE Trans. Circuits Syst. II Express Briefs.

[112]  Rudy Lauwereins,et al.  Code generation for compiled bit-true simulation of DSP applications , 1998, Proceedings. 11th International Symposium on System Synthesis (Cat. No.98EX210).

[113]  Romuald Rocher,et al.  Analytical Approach for Numerical Accuracy Estimation of Fixed-Point Systems Based on Smooth Operations , 2012, IEEE Transactions on Circuits and Systems I: Regular Papers.

[114]  Lothar Thiele,et al.  On the sensitivity of linear state-space systems , 1986 .

[115]  Daniel Ménard,et al.  Bit accurate roundoff noise analysis of fixed-point linear controllers , 2008, 2008 IEEE International Conference on Computer-Aided Control Systems.

[116]  V. Tavsanoglu,et al.  Optimal design of state - space digital filters by simultaneous minimization of sensitivity and roundoff noise , 1984 .

[117]  B. Widrow Statistical analysis of amplitude-quantized sampled-data systems , 1961, Transactions of the American Institute of Electrical Engineers, Part II: Applications and Industry.

[118]  Prithviraj Banerjee,et al.  Automatic conversion of floating point MATLAB programs into fixed point FPGA based hardware design , 2003, 11th Annual IEEE Symposium on Field-Programmable Custom Computing Machines, 2003. FCCM 2003..

[119]  Heinrich Meyr,et al.  FRIDGE: a fixed-point design and simulation environment , 1998, Proceedings Design, Automation and Test in Europe.

[120]  I. Hartimo,et al.  Elimination of zero-input and constant-input limit cycles in single-quantizer recursive filter structures , 1992 .

[121]  Kamal Premaratne,et al.  An exhaustive search algorithm for checking limit cycle behavior of digital filters , 1995, Proceedings of ISCAS'95 - International Symposium on Circuits and Systems.

[122]  A. Gray,et al.  Digital lattice and ladder filter synthesis , 1973 .

[123]  George A. Constantinides,et al.  Bounding Variable Values and Round-Off Effects Using Handelman Representations , 2011, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[124]  Vimal Singh An extension of Jury - Lee's criterion for the stability analysis of fixed-point digital filters designed with two's complement arithmetic , 1986 .

[125]  M. Gevers,et al.  Parametrizations in Control, Estimation and Filtering Problems: Accuracy Aspects , 1993 .

[126]  Holger Keding Pain Killers for the Fixed-Point Design Flow , 2010 .

[127]  Prithviraj Banerjee,et al.  Overview of a compiler for synthesizing MATLAB programs onto FPGAs , 2004, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[128]  W. Luk,et al.  Truncation noise in fixed-point SFGs [digital filters] , 1999 .

[129]  Jorge Stolfi,et al.  Affine Arithmetic: Concepts and Applications , 2004, Numerical Algorithms.

[130]  Juha Kauraniemi Analysis of limit cycles in the direct form delta operator structure by computer-aided test , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[131]  Paul D. Fiore,et al.  Efficient Approximate Wordlength Optimization , 2008, IEEE Transactions on Computers.

[132]  Seehyun Kim,et al.  A floating-point to fixed-point assembly program translator for the TMS 320C25 , 1994 .

[133]  L. Kristiansson,et al.  Necessary and sufficient conditions for the absence of overflow phenomena in a second-order recursive digital filter , 1975 .

[134]  Yvon Savaria,et al.  A comparison of automatic word length optimization procedures , 2002, 2002 IEEE International Symposium on Circuits and Systems. Proceedings (Cat. No.02CH37353).

[135]  W. Luk,et al.  Truncation noise in fixed-point SFGs , 1999 .

[136]  Chengshan Xiao Improved L2-sensitivity for state-space digital system , 1997, IEEE Trans. Signal Process..

[137]  Romuald Rocher,et al.  Accuracy Constraint Determination in Fixed-Point System Design , 2008, EURASIP J. Embed. Syst..

[138]  Philippe Chevrel,et al.  A Unifying Framework for Finite Wordlength Realizations , 2007, IEEE Transactions on Circuits and Systems I: Regular Papers.

[139]  Peter Y. K. Cheung,et al.  Word-length selection for power minimization via nonlinear optimization , 2009, TODE.

[140]  T. Hinamoto Minimization of L_2-sensitivity for state-space digital filters subject to L_2-dynamic-range scaling constraints , 2005 .

[141]  Wonyong Sung,et al.  Fixed-point C compiler for TMS320C50 digital signal processor , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[142]  De Figueiredo,et al.  Self-validated numerical methods and applications , 1997 .

[143]  Mark Stephenson,et al.  Bidwidth analysis with application to silicon compilation , 2000, PLDI '00.

[144]  Alok N. Choudhary,et al.  Precision and error analysis of MATLAB applications during automated hardware synthesis for FPGAs , 2001, Proceedings Design, Automation and Test in Europe. Conference and Exhibition 2001.