Applying Qualitative Comparative Analysis (QCA) to evaluate a public health policy initiative in the North East of England

Abstract This paper presents a Qualitative Comparative Analysis (QCA) analysis of data produced as part of the evaluation a Nation Health Service commissioned intervention in the North East of England. QCA is a case-oriented method that allows systematic comparison of cases as configurations of set memberships based on their attributes and the relationship of these to particular outcomes. QCA provides an alternative to conventional quantitative approaches which are generally concerned with isolating the independent effect of one variable whilst controlling the influence of others. Instead, QCA allows for interactions between multiple attributes and recognises that the same outcomes may be generated by different configurations of attributes. The intervention evaluated provided case management for individuals who were out of work due to ill health, and had been for three years or more. It aimed to improve the health of individuals and move them closer to the labour market. The intervention and a comparison group were assessed at base line (T1), after 3 months — (T2) after 6 months (end of the intervention — T3) and after 9 months (three months post intervention — T4). The size of the respective populations at each time point were, Intervention group at T1, N = 131, T2, N = 44, T3, N = 79, T4, N = 95. Comparison group at T1, N = 229, T2, N = 188, T3, N = 166, T4, N = 154. General health was measured using EQ5-D (a standardised instrument for use as a measure of general health outcome) and SF-8. Two condition specific measures were included: the Hospital Anxiety and Depression Scale (HADS) and the Nordic Musculoskeletal questionnaire. Data was also collected on socio demographics (gender, age, housing tenure), social capital (contact with family and friends and participation with the wider community), and work history (previous jobs, time spent in the job, time spent on sickness absence). The aim of the QCA analysis was to identify whether individuals with certain characteristics or combinations of characteristics benefited from the intervention. In order to do this the cases were sorted according to whether their EQ-5D VAS (Visual Analogue Scale) scores (a self rated measure of general health) narrowed or did not narrow towards the population norm for the measure between baseline (T1) and (T4) 9 months (three months post intervention — T4). Cases which narrowed towards the UK population norm of 82.48 were judged to be experiencing a health improvement whilst those whose scores did not narrow towards the norm were judged to not have experienced a health improvement. A crisp set (cs) QCA analysis was then performed. The paper assesses the benefits of using QCA, and asks whether it can provide a viable and practical tool for social policy evaluations.