Estimation with Quadratic Loss

It has long been customary to measure the adequacy of an estimator by the smallness of its mean squared error. The least squares estimators were studied by Gauss and by other authors later in the nineteenth century. A proof that the best unbiased estimator of a linear function of the means of a set of observed random variables is the least squares estimator was given by Markov [12], a modified version of whose proof is given by David and Neyman [4]. A slightly more general theorem is given by Aitken [1]. Fisher [5] indicated that for large samples the maximum likelihood estimator approximately minimizes the mean squared error when compared with other reasonable estimators. This paper will be concerned with optimum properties or failure of optimum properties of the natural estimator in certain special problems with the risk usually measured by the mean squared error or, in the case of several parameters, by a quadratic function of the estimators. We shall first mention some recent papers on this subject and then give some results, mostly unpublished, in greater detail.

[1]  R. Fisher,et al.  On the Mathematical Foundations of Theoretical Statistics , 1922 .

[2]  A. C. Aitken.  IV.—On Least Squares and Linear Combination of Observations , 1936 .

[3]  A. Wald.  Contributions to the Theory of Statistical Estimation and Testing Hypotheses , 1939 .

[4]  E. Pitman,et al.  THE ESTIMATION OF THE LOCATION AND SCALE PARAMETERS OF A CONTINUOUS POPULATION OF ANY GIVEN FORM , 1939 .

[5]  E. Pitman.  TESTS OF HYPOTHESES CONCERNING LOCATION AND SCALE PARAMETERS , 1939 .

[6]  D. Blackwell.  On the Translation Parameter Problem for Discrete Variables , 1951 .

[7]  L. J. Savage,et al.  Bayes and Minimax Estimates for Quadratic Loss Functions , 1951 .

[8]  C. Blyth.  ON MINIMAX STATISTICAL DECISION PROCEDURES AND THEIR ADMISSIBILITY , 1951 .

[9]  W. R. Buckland,et al.  Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability. , 1952 .

[10]  C. Stein.  A NECESSARY AND SUFFICIENT CONDITION FOR ADMISSIBILITY , 1955 .

[11]  Jerzy Neyman,et al.  Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability : held at the Statistical Laboratory, University of California, December, 1954, July and August, 1955 , 1958 .

[12]  R. Wijsman.  Random Orthogonal Transformations and their use in Some Classical Distribution Problems in Multivariate Analysis , 1957 .

[13]  J. Kiefer.  INVARIANCE, MINIMAX SEQUENTIAL ESTIMATION, AND CONTINUOUS TIME PROCESSES , 1957 .

[14]  S. Karlin.  Admissibility for Estimation with Quadratic Loss , 1958 .

[15]  C. Stein.  THE ADMISSIBILITY OF PITMAN'S ESTIMATOR OF A SINGLE LOCATION PARAMETER' , 1959 .

[16]  E. Lehmann.  Testing Statistical Hypotheses , 1960 .

[17]  W. Hoeffding,et al.  Contributions to Probability and Statistics: Essays in Honor of Harold Hotelling , 1961 .