The Method of Least Squares and Some Alternatives.

Abstract : A very important problem in mathematical statistics is that of finding the best linear or nonlinear regression equation to express the relation between a dependent variable and one or more independent variables. Given are observations, each subject to random error, greater in number than the parameters in the regression equation, on the dependent variable and the related values of the independent variable(s), which may be known exactly or may also be subject to random error. Related problems are those of choosing the best measures of central tendency and dispersion of the observations. The best solutions of all three problems depend upon the distribution of the random errors. If one assumes that the values of the independent variable(s) are known exactly and that the errors in the observations on the dependent variable are normally distributed, then it is well known that the mean is the best measure of central tendency, the standard deviation is the best measure of dispersion, and the method of least squares is the best method of fitting a regression equation. Other assumptions lead to different choices.