Software Numerical Instability Detection and Diagnosis by Combining Stochastic and Infinite-Precision Testing

Numerical instability is a well-known problem that may cause serious runtime failures. This paper discusses the reason of instability in software development process, and presents a toolchain that not only detects the potential instability in software, but also diagnoses the reason for such instability. We classify the reason of instability into two categories. When it is introduced by software requirements, we call the instability caused by problem . In this case, it cannot be avoided by improving software development, but requires inspecting the requirements, especially the underlying mathematical properties. Otherwise, we call the instability caused by practice. We design our toolchain as four loosely-coupled tools, which combine stochastic arithmetic with infinite-precision testing. Each tool in our toolchain can be configured with different strategies according to the properties of the analyzed software. We evaluate our toolchain on subjects from literature. The results show that it effectively detects and separates the instabilities caused by problems from others. We also conduct an evaluation on the latest version of GNU Scientific Library, and the toolchain finds a few real bugs in the well-maintained and widely deployed numerical library. With the help of our toolchain, we report the details and fixing advices to the GSL buglist.

[1]  Guillaume Melquiond,et al.  Flocq: A Unified Library for Proving Floating-Point Algorithms in Coq , 2011, 2011 IEEE 20th Symposium on Computer Arithmetic.

[2]  Matthieu Martel,et al.  Propagation of Roundoff Errors in Finite Precision Computations: A Semantics Approach , 2002, ESOP.

[3]  Webb Miller.  Software for Roundoff Analysis , 1975, TOMS.

[4]  Jean Vignes,et al.  Discrete Stochastic Arithmetic for Validating Results of Numerical Software , 2004, Numerical Algorithms.

[5]  Michael J. Schulte,et al.  Performance analysis of decimal floating-point libraries and its impact on decimal hardware and software solutions , 2009, 2009 IEEE International Conference on Computer Design.

[6]  Viktor Kuncak,et al.  Trustworthy numerical computation in Scala , 2011, OOPSLA '11.

[7]  Claude Marché,et al.  Multi-Prover Verification of Floating-Point Programs , 2010, IJCAR.

[8]  Matthieu Martel.  Semantics-Based Transformation of Arithmetic Expressions , 2007, SAS.

[9]  Christopher Kormanyos,et al.  Algorithm 910: A Portable C++ Multiple-Precision System for Special-Function Calculations , 2011, TOMS.

[10]  J. Vignes,et al.  Error Analysis in Computing , 1974, IFIP Congress.

[11]  Yuxiang Zhu,et al.  Stochastic Arithmetic in Multiprecision , 2011, Math. Comput. Sci..

[12]  Jens Blanck,et al.  Exact real arithmetic using centred intervals and bounded error terms , 2006, J. Log. Algebraic Methods Program..

[13]  Chein-Shan Liu,et al.  NOVEL METHODS FOR SOLVING SEVERELY ILL-POSED LINEAR EQUATIONS SYSTEM , 2009 .

[14]  Zhendong Su,et al.  Perturbing numerical calculations for statistical analysis of floating-point program (in)stability , 2010, ISSTA '10.

[15]  Miriam Leeser,et al.  Advanced Components in the Variable Precision Floating-Point Library , 2006, 2006 14th Annual IEEE Symposium on Field-Programmable Custom Computing Machines.

[16]  Matthieu Martel.  Program transformation for numerical precision , 2009, PEPM '09.

[17]  D. Stott Parker,et al.  Monte Carlo Arithmetic: a framework for the statistical analysis of roundo error , 1997 .

[18]  Eric Goubault,et al.  Static Analysis-Based Validation of Floating-Point Computations , 2003, Numerical Software with Result Verification.

[19]  erie M enissier-Morainy.  Arbitrary Precision Real Arithmetic: Design and Algorithms , 1996 .

[20]  Eric Goubault,et al.  Static Analysis of Numerical Algorithms , 2006, SAS.

[21]  Tarek A. El-Ghazawi,et al.  Bringing High-Performance Reconfigurable Computing to Exact Computations , 2007, 2007 International Conference on Field Programmable Logic and Applications.

[22]  Norbert Th. Müller,et al.  The iRRAM: Exact Arithmetic in C++ , 2000, CCA.

[23]  C. Willmott,et al.  Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance , 2005 .

[24]  Fabrice Rouillier,et al.  Motivations for an Arbitrary Precision Interval Arithmetic and the MPFI Library , 2005, Reliab. Comput..

[25]  Jean-Christophe Filliâtre,et al.  Formal Verification of Floating-Point Programs , 2007, 18th IEEE Symposium on Computer Arithmetic (ARITH '07).

[26]  James Hardy Wilkinson,et al.  Rounding errors in algebraic processes , 1964, IFIP Congress.

[27]  Keith Briggs,et al.  Implementing exact real arithmetic in python, C++ and C , 2006, Theor. Comput. Sci..

[28]  Mei Han An,et al.  accuracy and stability of numerical algorithms , 1991 .

[29]  Vincent Lefèvre,et al.  MPFR: A multiple-precision binary floating-point library with correct rounding , 2007, TOMS.

[30]  Zhendong Su,et al.  Osprey: a practical type system for validating dimensional unit correctness of C programs , 2006, ICSE.

[31]  Xiangyu Zhang,et al.  On-the-fly detection of instability problems in floating-point program execution , 2013, OOPSLA.

[32]  Alexander Stanoyevitch,et al.  Introduction to Partial Differential Equations , 2011, 1901.03022.

[33]  J. Stolfi,et al.  An Introduction to Affine Arithmetic , 2003 .

[34]  David L. Spooner,et al.  Software for roundoff analysis, II , 1978, TOMS.

[35]  Arnaud Gotlieb,et al.  Symbolic Path-Oriented Test Data Generation for Floating-Point Programs , 2013, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation.

[36]  David R. Kincaid,et al.  Numerical mathematics and computing , 1980 .

[37]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[38]  Zhendong Su,et al.  A Genetic Algorithm for Detecting Significant Floating-Point Inaccuracies , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[39]  Bronis R. de Supinski,et al.  Abstract: Automatically Adapting Programs for Mixed-Precision Floating-Point Computation , 2013, 2012 SC Companion: High Performance Computing, Networking Storage and Analysis.

[40]  Branimir Lambov.  RealLib: An efficient implementation of exact real arithmetic , 2007, Math. Struct. Comput. Sci..

[41]  Douglas Stott Parker,et al.  Perturbing and evaluating numerical programs without recompilation—the wonglediff way , 2005, Softw. Pract. Exp..

[42]  Eric Goubault,et al.  Static Analyses of the Precision of Floating-Point Operations , 2001, SAS.

[43]  Alexander Aiken,et al.  Stochastic optimization of floating-point programs with tunable precision , 2014, PLDI.

[44]  Miriam Leeser,et al.  VFloat: A Variable Precision Fixed- and Floating-Point Library for Reconfigurable Hardware , 2010, TRETS.

[45]  David L. Spooner,et al.  Automatic Generation of Floating-Point Test Data , 1976, IEEE Transactions on Software Engineering.

[46]  Jean Vignes,et al.  A stochastic arithmetic for reliable scientific computation , 1993 .

[47]  Zhendong Su,et al.  Automatic detection of floating-point exceptions , 2013, POPL.

[48]  Webb Miller,et al.  Toward mechanical verification of properties of roundoff error propagation , 1973, STOC.

[49]  D. Qainlant,et al.  ROSE: Compiler Support for Object-Oriented Frameworks , 1999 .

[50]  Sebastian Hack,et al.  A dynamic program analysis to find floating-point accuracy problems , 2012, PLDI.

[51]  James Demmel,et al.  Precimonious: Tuning assistant for floating-point precision , 2013, 2013 SC - International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[52]  D. Stott Parker,et al.  Monte Carlo Arithmetic: exploiting randomness in floating-point arithmetic , 1997 .

[53]  Xin-ping Xiao,et al.  Research on ill-conditioned problem and modeling precision in GM(1,1) model , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[54]  Douglas Stott Parker,et al.  Monte Carlo arithmetic: how to gamble with floating point and win , 2000, Comput. Sci. Eng..

[55]  Valérie Ménissier-Morain,et al.  Arbitrary precision real arithmetic: design and algorithms , 2005, J. Log. Algebraic Methods Program..

[56]  Fabienne Jézéquel,et al.  CADNA: a library for estimating round-off error propagation , 2008, Comput. Phys. Commun..

[57]  J. R. Marcial-Romero,et al.  Semantics of a sequential language for exact real-number computation , 2004, LICS 2004.