Multi-Campus Studies of College Impact: Which Statistical Method is Appropriate?

In most multi-campus studies of college impact that have been conducted over the past four decades, investigators have relied on ordinary least squares (OLS) regression as the analytic method of choice. Recently, however, some investigators have advocated the use of Hierarchical Linear Modeling (HLM), a method specifically designed for analyses that involve both individual (student) and aggregate (institutional) level measures. Cross-validation analyses using a national database show that the two methods yield an equally good “fit” with empirical data. Existing OLS software has the advantage of enabling one to perform path analytical causal modeling; HLM has the advantage of yielding a more conservative estimate of the significance of institution-level effects.

[1]  G. Bohrnstedt,et al.  Robustness in Regression Analysis , 1971 .

[2]  Bruce Thompson,et al.  Stepwise Regression and Stepwise Discriminant Analysis Need Not Apply here: A Guidelines Editorial , 1995 .

[3]  Jan de Leeuw,et al.  Random Coefficient Models for Multilevel Analysis , 1986 .

[4]  Anthony S. Bryk,et al.  A Hierarchical Model for Studying School Effects , 1986 .

[5]  Donald B. Rubin,et al.  Empirical bayes estimation of coefficients in the general linear model from data of deficient rank , 1983 .

[6]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[7]  D. Mackinnon,et al.  Multilevel Modeling of Individual and Group Level Mediated Effects , 2001, Multivariate behavioral research.

[8]  Yuk Fai Cheong,et al.  HLM 6: Hierarchical Linear and Nonlinear Modeling , 2000 .

[9]  Lee J. Cronbach,et al.  Research on Classrooms and Schools: Formulation of Questions, Design and Analysis. , 1976 .

[10]  G. W. Snedecor.  Statistical Methods , 1964 .

[11]  A. Astin,et al.  A Decade of Changes in Undergraduate Education: A National Study of System "Transformation" , 2002 .

[12]  Edgar F. Borgatta,et al.  Sociological Methodology, 1969. , 1969 .

[13]  Van Ark,et al.  Sociological Methodology 1971 , 1973 .

[14]  L. Poundie Burstein.  The Analysis of Multilevel Data in Educational Research and Evaluation , 1980 .

[15]  Alexander W. Astin,et al.  The American Freshman: National Norms for Fall 1982. , 1982 .

[16]  A. Astin,et al.  Assessment for Excellence: The Philosophy and Practice of Assessment and Evaluation in Higher Education. American Council on Education/Macmillan Series on Higher Education. , 1991 .

[17]  H. Goldstein.  Multilevel mixed linear model analysis using iterative generalized least squares , 1986 .

[18]  Lillian Cohen,et al.  Statistical Methods for Social Scientists. , 1954 .

[19]  Michael Seltzer.  Furthering Our Understanding of the Effects of Educational Programs via a Slopes-as-Outcomes Framework , 1995 .