The Effect of Estimating Weights in Weighted Least Squares

Abstract In weighted least squares, it is typical that the weights are unknown and must be estimated. Most packages provide standard errors assuming that the weights are known. This is fine for sufficiently large sample sizes, but what about for small-to-moderate sample sizes? The investigation of this article into the effect of estimating weights proceeds under the assumption typical in practice—that one has a parametric model for the variance function. In this context, generalized least squares consists of (a) an initial estimate of the regression parameter, (b) a method for estimating the variance function, and (c) the number of iterations in reweighted least squares. By means of expansions for the covariance, it is shown that each of (a)—(c) can matter in problems of small to moderate size. A few conclusions may be of practical interest. First, estimated standard errors assuming that the weights are known can be too small in practice. The investigation indicates that a simple bootstrap operation resul...

[1]  D. Freedman,et al.  Bootstrapping a Regression Equation: Some Empirical Results , 1984 .

[2]  Changbao Wu,et al.  Jackknife, Bootstrap and Other Resampling Methods in Regression Analysis , 1986 .

[3]  D. Freedman.  Bootstrapping Regression Models , 1981 .

[4]  Wayne A. Fuller,et al.  Least Squares Estimation When the Covariance Matrix and Parameter Vector are Functionally Related , 1980 .

[5]  P. McCullagh,et al.  Generalized Linear Models , 1984 .

[6]  D. Ruppert,et al.  Transformation and Weighting in Regression , 1988 .

[7]  Neil R. Ullman,et al.  Signal-to-noise ratios, performance criteria, and transformations , 1988 .

[8]  R. Carroll,et al.  Variance Function Estimation , 1987 .

[9]  K. Takeuchi,et al.  Asymptotic efficiency of statistical estimators : concepts and higher order asymptotic efficiency , 1981 .

[10]  W. Fuller,et al.  Estimation for a Linear Regression Model with Unknown Diagonal Covariance Matrix , 1978 .

[11]  B. Efron.  Bootstrap Methods: Another Look at the Jackknife , 1979 .

[12]  David Ruppert,et al.  Robust Estimation in Heteroscedastic Linear Models. , 1982 .

[13]  Richard E. Quandt,et al.  Nonlinear methods in econometrics , 1973 .

[14]  Thomas J. Rothenberg,et al.  APPROXIMATE NORMALITY OF GENERALIZED LEAST SQUARES ESTIMATES , 1984 .

[15]  Second-order expansion of mean squared error matrix of generalized least squares estimator with estimated parameters , 1982 .

[16]  George A. F. Seber,et al.  Linear regression analysis , 1977 .

[17]  A comparison of two methods for estimating optimal weights in regression analysis , 1984 .

[18]  D. Ruppert,et al.  A comparison between maximum likelihood and generalized least squares in a heteroscedastic linear model , 1982 .

[19]  Raymond J. Carroll,et al.  Adapting for Heteroscedasticity in Linear Models , 1982 .

[20]  Regina Y. Liu,et al.  On a Partial Correction by the Bootstrap , 1987 .