Multicollinearity: A Bayesian Interpretation

T HE problem of collinear data sets in the context of the univariate multiple regression model has generated a confusing array of papers, comments and footnotes. Perusal of this literature does not lead the reader to conclude that the problem has even been rigorously defined. The purpose of this paper is to offer several rigorous definitions and to suggest several substantive quantitative summaries of the degree of the problem. Specifically we address our attention to the regression model Y = X,B + u where ,B is a k-dimensional parameter vector with elements 81,12, ... *k, where Y and u are T X 1 vectors and where X is a T X k matrix. Inferences are to be made about the vector ,B from observations of Y and X. When the columns of X are orthogonal, the design matrix X'X is diagonal. Correlated columns of X imply a nondiagonal design matrix. The collinearity problem has to do with the differences in the inferences that may be drawn in these two situations. The principal claim of this paper is that the most important aspects of the collinearity problem derive from the existence of undominated uncertain prior information which causes major problems in interpreting the data evidence. It is claimed here that if our a priori knowledge of parameter values were either completely certain or "completely uncertain" the aspects of the collinearity problem that most of us worry about would disappear.' As an empirical test of this proposition consider the situations when collinearity is identified as a culprit. Usually signs are wrong or point estimates are otherwise peculiar. Occasionally confidence intervals overlap unlikely regions of the parameter space. Yet to say these things is to say there exists undominated uncertain prior information. Classical inference, with the possible exception of the pretesting literature, necessarily excludes undominated uncertain prior information. As a result most discussions of the collinearity problem miss a critical point. The textbook discussions including Theil (1971, p. 149), Malinvaud (1970, p. 218), and Goldberger (1964, p. 192), observe that when the design matrix X'X becomes singular, the least squares estimator is non-unique and the sampling distribution has finite variance only for certain "estimable" functions. Thus extreme collinearity is implicitly defined as total lack of sample information about some parameters. The case of less extreme collinearity is not dealt with so trivially since there is nothing in the least-squares theorems that is obviously dependent on the "near non-invertibility" of the design matrix. This fact has led Kmenta (1971, p. 391) to conclude "that a high degree of multicollinearity is simply a feature of the sample that contributes to the unreliability of the estimated coefficients, but has no relevance for the conclusions drawn as a result of this unreliability." To put this another way, the problem of defining collinearity may be solved by identifying a distance function for measuring the closeness of the design matrix to some noninvertible matrix in which the collinearity problem is unambiguously extreme. Since the extreme case is associated with infinite marginal variances on the parameters, authors such as Theil (1971, p. 152), Malinvaud (1970, p. 218), and Goldberger (1964, p. 193) use a distance function informally related to the sampling variance of the coefficients. Collinearity is defined as large variances. The failure of this definition is that instead of defining a new problem, it identifies a new cause of an already well-understood problem weak evidence. Although collinearity as a cause of the weak evidence problem can be distinguished from other causes such as small samples or large residual error variances, collinearity as Received for publication November 7, 1972. Revision accepted for publication January 23, 1973. * Research for this paper was supported by NSF Grant GS 319.29. The author has benefited from conversations on the subject with Gary Chamberlain and Richard Kopcke. Both the discussion and the content have benefited significantly from a referee's comments. An earlier version was presented at the NBER-NSF Seminar in Bayesian Inference in Econometrics at the University of Minnesota, October, 197,2. 1 See Zellner (1971, chapter 2) for a discussion of the problem of defining complete uncertainty.