Analysis of longitudinal data to evaluate a policy change

Longitudinal data analysis methods are powerful tools for exploring scientific questions regarding change and are well suited to evaluate the impact of a new policy. However, there are challenging aspects of policy change data that require consideration, such as defining comparison groups, separating the effect of time from that of the policy, and accounting for heterogeneity in the policy effect. We compare currently available methods to evaluate a policy change and illustrate issues specific to a policy change analysis via a case study of laws that eliminate gun-use restrictions (shall-issue laws) and firearm-related homicide. We obtain homicide rate ratios estimating the effect of enacting a shall-issue law, which vary between 0.903 and 1.101. We conclude that in a policy change analysis it is essential to select a mean model that most accurately characterizes the anticipated effect of the policy intervention, thoroughly model temporal trends, and select methods that accommodate unit-specific policy effects. We also conclude that several longitudinal data analysis methods are useful to evaluate a policy change, but not all may be appropriate in certain contexts. Analysts must carefully decide which methods are appropriate for their application and must be aware of the differences between methods to select a procedure that generates valid inference.

[1]  A K Wagner,et al.  Segmented regression analysis of interrupted time series studies in medication use research , 2002, Journal of clinical pharmacy and therapeutics.

[2]  Richard A Retting,et al.  Crash reduction following installation of centerline rumble strips on rural two-lane roads. , 2004, Accident; analysis and prevention.

[3]  N. Breslow,et al.  Approximate inference in generalized linear mixed models , 1993 .

[4]  Patrick J. Heagerty,et al.  Window Subsampling of Estimating Functions with Application to Regression Models , 2000 .

[5]  Gary K Grunwald,et al.  Analysis of repeated measures data with clumping at zero , 2002, Statistical methods in medical research.

[6]  T. Derouen,et al.  A Covariance Estimator for GEE with Improved Small‐Sample Properties , 2001, Biometrics.

[7]  P. Albert,et al.  Models for longitudinal data: a generalized estimating equation approach. , 1988, Biometrics.

[8]  M. Daniels,et al.  Hierarchical Generalized Linear Models in the Analysis of Variations in Health Care Utilization , 1999 .

[9]  N. Laird,et al.  Meta-analysis in clinical trials. , 1986, Controlled clinical trials.

[10]  S. Zeger,et al.  Longitudinal data analysis using generalized linear models , 1986 .

[11]  Douglas L. Miller,et al.  Does Head Start Improve Children's Life Chances? Evidence from a Regression Discontinuity Design , 2005, SSRN Electronic Journal.

[12]  S L Normand,et al.  Meta-analysis: formulating, evaluating, combining, and reporting. , 1999, Statistics in medicine.

[13]  Thomas B. Marvell,et al.  The Impact of “Shall-Issue” Concealed Handgun Laws on Violent Crime Rates , 2005 .

[14]  D W Webster,et al.  Flawed gun policy research could endanger public safety. , 1997, American journal of public health.

[15]  P Cummings,et al.  An evaluation of state firearm regulations and homicide and suicide death rates , 2005, Injury Prevention.

[16]  G. Casella.  An Introduction to Empirical Bayes Data Analysis , 1985 .

[17]  Roderick J. A. Little,et al.  Statistical Analysis with Missing Data: Little/Statistical Analysis with Missing Data , 2002 .

[18]  K C Carriere,et al.  Across time and space: variations in hospital use during Canadian health reform. , 2000, Health services research.

[19]  R A Deyo,et al.  Surgical and nonsurgical management of lumbar spinal stenosis: four-year outcomes from the maine lumbar spine study. , 2000, Spine.

[20]  K. Simon,et al.  How did welfare reform affect the health insurance coverage of women and children? , 2006, Health services research.

[21]  S. Piantadosi,et al.  The ecological fallacy. , 1988, American journal of epidemiology.

[22]  David B. Mustard,et al.  Crime, Deterrence, and Right‐to‐Carry Concealed Handguns , 1997, The Journal of Legal Studies.