Using qualitative metasummary to synthesize empirical findings in literature reviews

Context- A common problem in Systematic Reviews in software engineering is that they provide very limited syntheses. Goal- In the search for alternatives of effective methods for synthesizing empirical evidence, in this paper, we explore the use of the Qualitative Metasummary method, which is a quantitatively oriented aggregation of mixed research findings. Method - We describe the use of qualitative metasummary through an example using 15 studies addressing antecedents of performance of software development teams. Qualitative metasummary includes extraction and grouping of findings, and calculation of frequency and intensity effect sizes. Results -- The instance described in this paper produced a 10-factor model that effectively summarizes the current empirical knowledge on performance of software development teams. Then, we assessed the method in terms of ease of use, usefulness and reliability of results. Conclusion -- The Qualitative Metasummary method offers rich indexes of experiences and events under investigation, focusing on the effects of a variable over other, which is consistent with the central interest of systematic reviews. However, its main limitations are (i) challenging comparability/integratability between primary studies, (ii) loss of detailed contextual information, (iii) and the great deal of effort demanded to synthesize larger sets of papers.