The importance of conducting and reporting pilot studies: the example of the Scottish Births Survey.

BACKGROUND In many research papers, pilot studies are only reported as a means of justifying the methods. This justification might refer to the overall research design, or simply to the validity and reliability of the research tools. It is unusual for reports of pilot studies to include practical problems faced by the researcher(s). Pilot studies are relevant to best practice in research, but their potential for other researchers appears to be ignored. OBJECTIVE The primary aim of this study was to identify the most appropriate method for conducting a national survey of maternity care. METHODS Pilot studies were conducted in five hospitals to establish the best of four possible methods of approaching women, distributing questionnaires and encouraging the return of these questionnaires. Variations in the pilot studies included (a) whether or not the questionnaires were anonymous, (b) the staff involved in distributing the questionnaires and (c) whether questionnaires were distributed via central or local processes. For this purpose, five maternity hospitals of different sizes in Scotland were included. RESULTS Problems in contacting women as a result of changes in the Data Protection Act (1998) required us to rely heavily on service providers. However, this resulted in a number of difficulties. These included poor distribution rates in areas where distribution relied upon service providers, unauthorized changes to the study protocol and limited or inaccurate information regarding the numbers of questionnaires distributed. CONCLUSIONS The pilot raised a number of fundamental issues related to the process of conducting a large-scale survey, including the method of distributing the questionnaire, gaining access to patients, and reliance on 'gatekeepers'. This paper highlights the lessons learned as well as the balancing act of using research methods in the most optimal way under the combined pressure of time, ethical considerations and the influences of stakeholders. Reporting the kinds of practical issues that occur during pilot studies might help others avoid similar pitfalls and mistakes.

[1]  L. Curtice,et al.  Disseminating research results-the challenge of bridging the gap between health research and health action , 1994 .

[2]  A. Prevost,et al.  Psychological effect of witnessed resuscitation on bereaved relatives , 1998, The Lancet.

[3]  C. Baird,et al.  The pilot study. , 2000, Orthopedic nursing.

[4]  K. Barriball,et al.  Non-response in survey research: a methodological discussion and development of an explanatory model. , 1999, Journal of advanced nursing.

[5]  R. Lindquist Don't forget the pilot work! , 1991, Heart & lung : the journal of critical care.

[6]  W. Graham,et al.  Women's priorities for care before and after delivery , 1998 .

[7]  E. Thornton NLN Press Primary Health Care in Urban Communities , 2001 .

[8]  A. Hulst Dutch midwives: relational care and birth location. , 1999 .

[9]  M. Drummond,et al.  The Role of Pilot Studies in the Economic Evaluation of Health Technologies , 1998, International Journal of Technology Assessment in Health Care.

[10]  I. Kant,et al.  Dwalingen in de methodologie. VIII. Pilotonderzoeken: Zin en onzin , 1998 .

[11]  A. Wilson,et al.  Differentiating between audit and research: postal survey of health authorities' views , 1999, BMJ.

[12]  N. Black,et al.  The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. , 1998, Journal of epidemiology and community health.

[13]  E. Breeze,et al.  Design, objectives, and lessons from a pilot 25 year follow up re-survey of survivors in the Whitehall study of London Civil Servants. , 1998, Journal of epidemiology and community health.

[14]  W. Graham,et al.  A national survey of women's views of their maternity care in Scotland. , 2000, Midwifery.

[15]  P. Prescott,et al.  The potential uses of pilot work. , 1989, Nursing research.

[16]  V. Hundley,et al.  Satisfaction and the three C's: continuity, choice and control. Women's views from a randomised controlled trial of midwife‐led care , 1997, British journal of obstetrics and gynaecology.

[17]  Margaret Thom,et al.  Great Expectations: a Prospective Study of Women's Expectations and Experiences , 1999 .

[18]  R. Baker,et al.  Development of a questionnaire to assess patients' satisfaction with consultations in general practice. , 1990, The British journal of general practice : the journal of the Royal College of General Practitioners.

[19]  C. A. Moser,et al.  Survey Methods in Social Investigation , 1958 .