Applying Hierarchical Linear Modeling to Extended Longitudinal Evaluations

Longitudinal research designs with many waves of data have the potential to provide a fine-grained description of program impact, so they should be of special value for evaluation research. This potential has been illusive because our principal analysis methods are poorly suited to the task. We present strategies for analyzing these designs using hierarchical linear modeling (HLM). The basic growth curve models found in most longitudinal applications of HLM are not well suited to program evaluation, so we develop more appropriate alternatives. Our approach defines well-focused parameters that yield meaningful effect-size estimates and significance tests, efficiently combining all waves of data available for each subject. These methods do not require a uniform set of observations from all respondents. The Boys Town Follow-Up Study, an exceptionally rich but complex data set, is used to illustrate our approach.

[1]  G. R. Davies,et al.  The Growth Curve , 1927 .

[2]  Barbara Entwisle,et al.  Contextual analysis through the multilevel linear model. , 1983 .

[3]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[4]  D. Osgood,et al.  Use of pooled time series in the study of naturally occurring clinical events and problem behavior in a foster care setting. , 1994, Journal of consulting and clinical psychology.

[5]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[6]  David Rauma,et al.  Estimation Procedures for Pooled Cross-Sectional and Time Series Data , 1979 .

[7]  H. Goldstein,et al.  Multilevel Models in Educational and Social Research. , 1989 .

[8]  D. Rubin,et al.  The central role of the propensity score in observational studies for causal effects , 1983 .

[9]  Stephen W. Raudenbush,et al.  Growth Curve Analysis in Accelerated Longitudinal Designs , 1992 .

[10]  Cheng Hsiao,et al.  Analysis of Panel Data , 1987 .

[11]  James J. Heckman,et al.  Causal Inference and Nonrandom Samples , 1989 .

[12]  John B. Willett,et al.  Questions and Answers in the Measurement of Change , 1988 .

[13]  N M Thompson,et al.  Analysis of change: modeling individual growth. , 1991, Journal of consulting and clinical psychology.

[14]  Delbert S. Elliott,et al.  Explaining Delinquency and Drug Use , 1985 .

[15]  M. Hardy.  Regression with dummy variables , 1993 .

[16]  A. Bryk,et al.  Early vocabulary growth: Relation to language input and gender. , 1991 .

[17]  E. Pedhazur.  Multiple Regression in Behavioral Research: Explanation and Prediction , 1982 .

[18]  David Rogosa,et al.  A growth curve approach to the measurement of change. , 1982 .

[19]  J. Schumaker,et al.  Achievement place: The teaching-family model , 1976 .

[20]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[21]  Larry E. Toothaker,et al.  Multiple Regression: Testing and Interpreting Interactions , 1991 .

[22]  Anthony S. Bryk,et al.  Application of Hierarchical Linear Models to Assessing Change , 1987 .

[23]  G. Vining,et al.  Data Analysis: A Model-Comparison Approach , 1989 .