Concerns about Methodology Used in Real-World Experiments on Transport and Transport Safety

Real-world experiments in which some experimental units (e.g., people or roadway sites or vehicles) are randomized to treatment and others to no treatment are increasingly regarded as practicable and desirable. This is the case for complex educational and social welfare interventions as well as for therapeutic drug trials. The present paper discusses the relevance of such experiments to transport and transport safety research. When compared with the more common designs of research, both the advantages and the disadvantages may seem substantial, and decisions will require good judgment. Drawing lessons from medical and social welfare research, the paper aims to be a resource when the relative merits of randomized experimentation and other methods are being debated.

[1]  James V. Musick Effect of pavement edge marking on two-lane rural state highways in Ohio , 1960 .

[2]  Donald W. Reinfurt,et al.  ACCIDENT RESEARCH MANUAL , 1980 .

[3]  Ezra Hauer,et al.  The harm done by tests of significance. , 2004, Accident; analysis and prevention.

[4]  James J. Heckman,et al.  Assessing the Case for Social Experiments , 1995 .

[5]  Gary Burtless,et al.  The Case for Randomized Field Trials in Economic and Policy Research , 1995 .

[6]  R. Prescott,et al.  Factors that limit the quality, number and progress of randomised controlled trials. , 1999, Health technology assessment.

[7]  Ray Pawson,et al.  LEARNING LESSONS: ENHANCING EVALUATION THROUGH RESEARCH REVIEW , 2001 .

[8]  Andrew Leigh Randomised Policy Trials , 2003 .

[9]  Janet Wittes Randomized Treatment Assignment , 2005 .

[10]  S. L. Mann,et al.  Experimental design and methods for school-based randomized trials. Experience from the Hutchinson Smoking Prevention Project (HSPP). , 2000, Controlled clinical trials.

[11]  P. Gøtzsche,et al.  Blinding during data analysis and writing of manuscripts. , 1996, Controlled clinical trials.

[12]  Catherine Pope,et al.  Resisting Evidence: The Study of Evidence-Based Medicine as a Contemporary Social Movement , 2003 .

[13]  M. Bird Sheila,et al.  Criminal justice on trial , 2004 .

[14]  A J Basile EFFECT OF PAVEMENT EDGE MARKINGS ON TRAFFIC ACCIDENTS IN KANSAS , 1962 .

[15]  J. Grossman,et al.  Evaluating social policies : principles and U.S. experience , 1994 .

[16]  L. Watson,et al.  Mounting a community-randomized trial: sample size, matching, selection, and randomization issues in PRISM. , 2004, Controlled clinical trials.

[17]  I. Roberts,et al.  Evidence based road safety: the Driving Standards Agency's schools programme , 2001, The Lancet.

[18]  A. Feinstein,et al.  Problems in the "evidence" of "evidence-based medicine". , 1997, The American journal of medicine.

[19]  Franklyn W. Dunford Random assignment: Practical considerations from field experiments , 1990 .

[20]  Nick Tilley,et al.  Caring Communities, Paradigm Polemics, Design Debates , 1998 .

[21]  David P. Farrington,et al.  British Randomized Experiments on Crime and Justice , 2003 .

[22]  Anne Whitehead,et al.  Meta-Analysis of Controlled Clinical Trials , 2002 .

[23]  Paul J. Gertler,et al.  Using Randomized Control Designs in Evaluating Social Sector Programs in Developing Countries , 1994 .

[24]  L. Sherman Misleading Evidence and Evidence-Led Policy: Making Social Science more Experimental , 2003 .

[25]  F. Mosteller,et al.  Evidence matters : randomized trials in education research , 2002 .

[26]  Julie van den Eynde,et al.  They look good but don't work: a case study of global performance indicators in crime prevention , 2003 .

[27]  Alex J. Sutton,et al.  Methods for Meta-Analysis in Medical Research , 2000 .

[28]  Rune Elvik,et al.  The Handbook of Road Safety Measures , 2009 .

[29]  Alan Shiell,et al.  Complex interventions: how “out of control” can a randomised controlled trial be? , 2004, BMJ : British Medical Journal.

[30]  Rahim F Benekohal,et al.  PROCEDURES FOR ESTIMATING ACCIDENT REDUCTIONS ON TWO-LANE HIGHWAYS , 1992 .

[31]  Carol Taylor Fitz-Gibbon,et al.  Editorial: The need for randomized trials in social research , 2004 .

[32]  Meg Wiggins,et al.  Using Random Allocation to Evaluate Social Interventions: Three Recent U.K. Examples , 2003 .

[33]  David P. Farrington,et al.  Evaluating ‘Communities that Care’ , 1998 .

[34]  Timothy Paul Hutchinson,et al.  Evidence-based road safety policy? Evidence-based transport policy? A discussion of randomised experimentation and meta-analysis in the evaluation of interventions , 2004 .

[35]  Howard S. Bloom,et al.  Extending the reach of randomized social experiments: new directions in evaluations of American welfare‐to‐work and employment initiatives , 2001 .

[36]  John C Glennon,et al.  Evaluation of Minor Improvements , 1968 .

[37]  Joseph E. Hummer Procedures for Estimating Accident Reductions on Two-Lane Highways -- Discussion , 1993 .

[38]  Simon G. Thompson,et al.  Meta‐Analysis of Clinical Trials , 2005 .