On properties of floating point arithmetics: numerical stability and the cost of accurate computations

Floating point arithmetics generally possess many regularity properties in addition to those that are typically used in roundoff error analyses; these properties can be exploited to produce computations that are more accurate and cost effective than many programmers might think possible. Furthermore, many of these properties are quite simple to state and to comprehend, but few programmers seem to be aware of them (or at least willing to rely on them). This dissertation presents some of these properties and explores their consequences for computability, accuracy, cost, and portability. For example, we consider several algorithms for summing a sequence of numbers and show that under very general hypotheses, we can compute a sum to full working precision at only somewhat greater cost than a simple accumulation, which can often produce a sum with no significant figures at all. This example, as well as others we present, can be generalized further by substituting still more complex algorithms; consequently, examples such as these oblige us to consider more carefully the tradeoffs between cost and accuracy. At one end of the accuracy spectrum we find one of the least obvious consequences of the properties of floating point arithmetic: the accuracy of a computation consisting of rational arithmetic operations and comparisons need not be limited by the precision of the floating point arithmetic in which it is carried out. Of course, the more accuracy desired, the greater the cost of the computation, and the cost of computing a very accurate result may be quite high; we illustrate this possibility in the case of polynomial evaluation. At the other end of the spectrum, however, we give an example of a problem for which simply computing a result to a modest guaranteed accuracy costs far less than the contortions required to accommodate inaccurate results. As a consequence of examples such as these, we conclude that programmers and theorists alike must be willing to adopt a more sophisticated view of floating point arithmetic, even if only to consider that more accurate and reliable computations than those presently in common use might be possible based on stronger hypotheses than are customarily assumed.