Curve Fitting with Least Squares

Least-squares analysis is based on the socalled ‘normal’ distribution of experimental uncertainties formulated by Abraham de Moivre, a friend of Newton. De Moivre was a mathematical genius who never found permanent employment, and earned a meager living in London as a math tutor for the children of the wealthy. The least squares method itself was subsequently developed by Legendre and Gauss, who realized how the ‘normal’ distribution could be used for data analysis. For a long time, least squares analysis was the near-exclusive domain of statisticians, who used long tables to evaluate their data. The field advanced through the introduction of matrix algebra, although that tended to make it less accessible to non-statisticians. More recently, these matrix methods were implemented in specialized software and, finally, in generalpurpose computational programs such as spreadsheets. Now that they are easily accessible, least squares methods have become ubiquitous in science. In Section II we review some of the properties of the normal distribution and related concepts. We also see how multiple, small, random errors that each follow a ‘normal’ distribution lead to the least squares criterion. In Section III we then apply this method in order to fit experimental data to model expressions. The mathematics of least squares are greatly simplified when we can assume that the experimental deviations are essentially confined to one parameter, the ‘dependent’ variable. (We will make this simplifying assumption throughout this review, except in Section VII.) There can be any number of ‘independent’ variables, which assumedly do not contribute to the experimental uncertainty. The mathematics are especially simple with a single ‘independent’ variable that is uniformly spaced, as in the special case of equidistant data discussed in Section IV. Weighted least squares are discussed in Section V. The ready availability of multi-parameter non-linear least squares routines in modern spreadsheets and computerbased statistical packages provides yet another approach to curve fitting, as described in Section VI. And in Section VII we finally lift the constraining assumption that only one parameter contains random experimental fluctuations. Least-squares have found many other uses, such as to discover the presence (or absence) of linear relations between parameters (expressed in terms of the often misapplied linear correlation coefficient r), or to find the more subtle linear combinations of data that form the basis of near-infrared spectrometric analysis. For the latter, the reader should consult books on partial least squares or principal component analysis, since they fall outside the purview of the present review.