Protocol for an Empirical Study of Structured Abstracts

Background. We have been undertaking a series of systematic literature reviews as part of a research program aimed at assessing evidence-based software engineering. Medical standards for systematic reviews suggest that initial searches of candidate primary studies can rely on the title and the abstract to determine the eligibility of primary studies. Our experience indicates that software engineering abstracts are of such poor quality that it is often impossible to assess the eligibility of a primary study without reading some parts of the study itself. Medicine and psychology recommend the use of structure abstracts to improve the quality of abstracts. There have been several experimental studies of structured abstracts that have confirmed their value in psychology. Objective. This protocol defines a research plan aimed at assessing whether structured abstracts exhibit better structural properties than conventional abstracts for software engineering articles. Method. Conventional abstracts from two published proceedings of the EASE conference will be rewritten as structured abstracts. Metrics such as length, average sentence length, the Flesch readability index and amount and nature of added information will be obtained from different versions of the abstracts. The metrics for conventional and structured abstracts will be compared. Resource requirements. The experiment will require a research supervisor and two student researchers for a period of 8 weeks, and, in addition, support from other experienced research staff from the EBSE project.