What is actually measured in process evaluations for worksite health promotion programs: a systematic review

BackgroundNumerous worksite health promotion program (WHPPs) have been implemented the past years to improve employees’ health and lifestyle (i.e., physical activity, nutrition, smoking, alcohol use and relaxation). Research primarily focused on the effectiveness of these WHPPs. Whereas process evaluations provide essential information necessary to improve large scale implementation across other settings. Therefore, this review aims to: (1) further our understanding of the quality of process evaluations alongside effect evaluations for WHPPs, (2) identify barriers/facilitators affecting implementation, and (3) explore the relationship between effectiveness and the implementation process.MethodsPubmed, EMBASE, PsycINFO, and Cochrane (controlled trials) were searched from 2000 to July 2012 for peer-reviewed (randomized) controlled trials published in English reporting on both the effectiveness and the implementation process of a WHPP focusing on physical activity, smoking cessation, alcohol use, healthy diet and/or relaxation at work, targeting employees aged 18-65 years.ResultsOf the 307 effect evaluations identified, twenty-two (7.2%) published an additional process evaluation and were included in this review. The results showed that eight of those studies based their process evaluation on a theoretical framework. The methodological quality of nine process evaluations was good. The most frequently reported process components were dose delivered and dose received. Over 50 different implementation barriers/facilitators were identified. The most frequently reported facilitator was strong management support. Lack of resources was the most frequently reported barrier. Seven studies examined the link between implementation and effectiveness. In general a positive association was found between fidelity, dose and the primary outcome of the program.ConclusionsProcess evaluations are not systematically performed alongside effectiveness studies for WHPPs. The quality of the process evaluations is mostly poor to average, resulting in a lack of systematically measured barriers/facilitators. The narrow focus on implementation makes it difficult to explore the relationship between effectiveness and implementation. Furthermore, the operationalisation of process components varied between studies, indicating a need for consensus about defining and operationalising process components.

[1]  Patricia van Assema,et al.  Gezondheidsvoorlichting en gedragsverandering , 2000 .

[2]  A. J. van der Beek,et al.  What are possible barriers and facilitators to implementation of a Participatory Ergonomics programme? , 2010, Implementation science : IS.

[3]  Mark G. Wilson,et al.  Second-Year Results of an Obesity Prevention Program at The Dow Chemical Company , 2009, Journal of occupational and environmental medicine.

[4]  T. Yap,et al.  Tailored e-mails in the workplace. , 2010, AAOHN journal : official journal of the American Association of Occupational Health Nurses.

[5]  W. Kirsten Making the link between health and productivity at the workplace--a global perspective. , 2010, Industrial health.

[6]  K. Hansen,et al.  Implementation of physical coordination training and cognitive behavioural training interventions at cleaning workplaces – secondary analyses of a randomised controlled trial , 2012, Ergonomics.

[7]  A. J. van der Beek,et al.  A worksite vitality intervention to improve older workers' lifestyle and vitality-related outcomes: results of a randomised controlled trial , 2012, Journal of Epidemiology & Community Health.

[8]  Allard J van der Beek,et al.  Participatory ergonomics to reduce exposure to psychosocial and physical risk factors for low back pain and neck pain: results of a cluster randomised controlled trial , 2010, Occupational and Environmental Medicine.

[9]  C. Pope,et al.  Qualitative Research in Health Care , 1999 .

[10]  Kristy Sanderson,et al.  Process Evaluation in Occupational Stress Management Programs: A Systematic Review , 2007, American journal of health promotion : AJHP.

[11]  Wei Pan,et al.  The Effect of Tailored E-mails in the Workplace , 2009, AAOHN journal : official journal of the American Association of Occupational Health Nurses.

[12]  Jennie D. Bowen,et al.  First-Year Results of an Obesity Prevention Program at The Dow Chemical Company , 2009, Journal of occupational and environmental medicine.

[13]  C. Abraham,et al.  A taxonomy of behavior change techniques used in interventions. , 2008, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[14]  May H. Yang,et al.  A process evaluation of tobacco-related outcomes from a telephone and print-delivered intervention for motor freight workers. , 2010, Addictive behaviors.

[15]  P. Fagan,et al.  Reducing Cigarette Smoking Among Working Adolescents: Results from the SMART Study , 2005, Cancer Causes & Control.

[16]  T. Kristensen Intervention studies in occupational epidemiology , 2005, Occupational and Environmental Medicine.

[17]  M. Heymans,et al.  Small portion sizes in worksite cafeterias: do they help consumers to reduce their food intake? , 2011, International Journal of Obesity.

[18]  Claire Bombardier,et al.  2009 Updated Method Guidelines for Systematic Reviews in the Cochrane Back Review Group , 2009, Spine.

[19]  David M DeJoy,et al.  Move to Improve: a randomized workplace trial to increase physical activity. , 2009, American journal of preventive medicine.

[20]  A. Stoddard,et al.  Process Evaluation Results From the Healthy Directions–Small Business Study , 2007, Health education & behavior : the official publication of the Society for Public Health Education.

[21]  M. Petticrew,et al.  Developing and evaluating complex interventions: the new Medical Research Council guidance , 2008, BMJ : British Medical Journal.

[22]  Kevin G Volpp,et al.  A randomized, controlled trial of financial incentives for smoking cessation. , 2009, The New England journal of medicine.

[23]  Annice E Kim,et al.  Why are Financial Incentives not Effective at Influencing Some Smokers to Quit? Results of a Process Evaluation of a Worksite Trial Assessing the Efficacy of Financial Incentives for Smoking Cessation , 2011, Journal of occupational and environmental medicine.

[24]  Carlton Cooke,et al.  Experiences of route and task-based walking in a university community: qualitative perspectives in a randomized control trial. , 2008, Journal of physical activity & health.

[25]  Sue Ziebland,et al.  Analysing qualitative data , 2000, BMJ : British Medical Journal.

[26]  S. Lemon,et al.  Evaluating the Implementation of a Hospital Work-Site Obesity Prevention Intervention , 2012, Health promotion practice.

[27]  May H. Yang,et al.  Tobacco use cessation and weight management among motor freight workers: results of the gear up for health study , 2010, Cancer Causes & Control.

[28]  Allard J van der Beek,et al.  Sustained body weight reduction by an individual-based lifestyle intervention for workers in the construction industry at risk for cardiovascular disease: results of a randomized controlled trial. , 2010, Preventive medicine.

[29]  Lisa M Klesges,et al.  The future of health behavior change research: What is needed to improve translation of research into health promotion practice? , 2004, Annals of behavioral medicine : a publication of the Society of Behavioral Medicine.

[30]  Allard J van der Beek,et al.  The Effectiveness of Worksite Physical Activity Programs on Physical Activity, Physical Fitness, and Health , 2003, Clinical journal of sport medicine : official journal of the Canadian Academy of Sport Medicine.

[31]  R. Glasgow,et al.  Evaluating the public health impact of health promotion interventions: the RE-AIM framework. , 1999, American journal of public health.

[32]  Stephen Platt,et al.  The effectiveness of interventions to change six health behaviours: a review of reviews , 2010, BMC public health.

[33]  Cliona Ni Mhurchu,et al.  Effects of worksite health promotion interventions on employee diets: a systematic review , 2010, BMC public health.

[34]  L. Andersen,et al.  Effectiveness of small daily amounts of progressive resistance training for frequent neck/shoulder pain: Randomised controlled trial , 2011, PAIN®.

[35]  William B. Cissell,et al.  Process Evaluation for Public Health Interventions and Research , 2004 .

[36]  Victoria A. Andersen,et al.  Step ahead a worksite obesity prevention trial among hospital employees. , 2010, American journal of preventive medicine.

[37]  Ruth P Saunders,et al.  Developing a Process-Evaluation Plan for Assessing Health Promotion Program Implementation: A How-To Guide , 2005, Health promotion practice.

[38]  H. Hasson,et al.  Factors associated with high use of a workplace web-based stress management program in a randomized controlled intervention study. , 2010, Health education research.

[39]  David M DeJoy,et al.  Do intervention fidelity and dose influence outcomes? Results from the move to improve worksite physical activity program. , 2010, Health education research.

[40]  E. Rogers,et al.  Diffusion of innovations , 1964, Encyclopedia of Sport Management.

[41]  K. Wiefferink,et al.  Invoering van effectief gebleken interventies , 2007 .

[42]  Allard J van der Beek,et al.  Effectiveness of physical activity programs at worksites with respect to work-related outcomes. , 2002, Scandinavian journal of work, environment & health.

[43]  E. Rogers,et al.  Diffusion of Innovations , 1964 .

[44]  J. Durlak,et al.  Implementation Matters: A Review of Research on the Influence of Implementation on Program Outcomes and the Factors Affecting Implementation , 2008, American journal of community psychology.

[45]  K. Cameron,et al.  Diagnosing and Changing Organizational Culture: Based on the Competing Values Framework , 1999 .

[46]  E. Viikari-Juntura,et al.  Effect of a participatory ergonomics intervention on psychosocial factors at work in a randomised controlled trial , 2009, Occupational and Environmental Medicine.

[47]  A. Burdorf,et al.  International Journal of Behavioral Nutrition and Physical Activity Determinants of Participation in Worksite Health Promotion Programmes: a Systematic Review , 2009 .

[48]  Elizabeth Allen,et al.  Process evaluation in randomised controlled trials of complex interventions , 2006, BMJ : British Medical Journal.

[49]  Candice Bowman,et al.  The role of formative evaluation in implementation research and the QUERI experience , 2006, Journal of General Internal Medicine.

[50]  A. J. van der Beek,et al.  A process evaluation of a worksite vitality intervention among ageing hospital workers , 2011, The international journal of behavioral nutrition and physical activity.

[51]  David M. DeJoy,et al.  Management Support and Worksite Health Promotion Program Effectiveness , 2009, HCI.

[52]  W. van Mechelen,et al.  The application of an occupational health guideline reduces sedentary behaviour and increases fruit intake at work: results from an RCT , 2012, Occupational and Environmental Medicine.

[53]  Karina Nielsen,et al.  Conducting organizational-level occupational health interventions: What works? , 2010 .

[54]  M. P. Jans,et al.  Overweight and Obesity as Predictors of Absenteeism in the Working Population of the Netherlands , 2007, Journal of occupational and environmental medicine.

[55]  Brian S Mittman,et al.  Creating the Evidence Base for Quality Improvement Collaboratives , 2004, Annals of Internal Medicine.

[56]  I. Steenhuis,et al.  Process evaluation of two environmental nutrition programmes and an educational nutrition programme conducted at supermarkets and worksite cafeterias in the Netherlands. , 2004, Journal of human nutrition and dietetics : the official journal of the British Dietetic Association.

[57]  J. Seidell,et al.  The process evaluation of two interventions aimed at portion size in worksite cafeterias. , 2012, Journal of human nutrition and dietetics : the official journal of the British Dietetic Association.

[58]  I. Steenhuis,et al.  The impact of educational and environmental interventions in Dutch worksite cafeterias. , 2004, Health promotion international.

[59]  W. van Mechelen,et al.  Process Evaluation of an Occupational Health Guideline Aimed at Preventing Weight Gain Among Employees , 2011, Journal of occupational and environmental medicine.

[60]  D. McLerran,et al.  Seattle 5-a-Day Work-Site Project: Process Evaluation , 2000, Health education & behavior : the official publication of the Society for Public Health Education.

[61]  P. Fagan,et al.  Feasibility of implementing intervention methods in an adolescent worksite tobacco control study , 2003, Tobacco control.

[62]  Anne M Stoddard,et al.  Promoting behavior change among working-class, multiethnic workers: results of the healthy directions--small business study. , 2005, American journal of public health.

[63]  J. Geddes,et al.  What is a randomised controlled trial? , 2009, Epidemiologia e Psichiatria Sociale.

[64]  Mark G. Wilson,et al.  Using Formative Research to Develop Environmental and Ecological Interventions to Address Overweight and Obesity , 2007, Obesity.

[65]  A Burdorf,et al.  Determinants of implementation of primary preventive interventions on patient handling in healthcare: a systematic review , 2009, Occupational and Environmental Medicine.

[66]  Carlton Cooke,et al.  Walking towards health in a university community: a feasibility study. , 2007, Preventive medicine.

[67]  M. J. Macarthur,et al.  Elements of External Validity of Tools for Health: An Intervention for Construction Laborers , 2010, American journal of health promotion : AJHP.

[68]  L. Andersen Influence of Psychosocial Work Environment on Adherence to Workplace Exercise , 2011, Journal of occupational and environmental medicine.

[69]  W. van Mechelen,et al.  Barriers and Facilitators to Implementation of an Occupational Health Guideline Aimed at Preventing Weight Gain Among Employees in the Netherlands , 2012, Journal of occupational and environmental medicine.

[70]  D McLerran,et al.  Seattle 5 a Day worksite program to increase fruit and vegetable consumption. , 2001, Preventive medicine.

[71]  Tuija Virtanen,et al.  Evaluation of a participatory ergonomic intervention process in kitchen work. , 2009, Applied ergonomics.

[72]  Willem van Mechelen,et al.  Worksite health promotion programs with environmental changes: a systematic review. , 2005, American journal of preventive medicine.

[73]  Allard J van der Beek,et al.  Process evaluation of a participatory ergonomics programme to prevent low back pain and neck pain among workers , 2010, Implementation science : IS.

[74]  A. DiCenso,et al.  Factors affecting nurse practitioner role implementation in Canadian practice settings: an integrative review. , 2011, Journal of advanced nursing.

[75]  Joel Tsevat,et al.  Modeling the effects of spirituality/religion on patients’ perceptions of living with HIV/AIDS , 2006, Journal of General Internal Medicine.

[76]  A. Burdorf,et al.  Productivity loss in the workforce: associations with health, work demands, and individual characteristics. , 2009, American journal of industrial medicine.

[77]  B. Arnetz,et al.  Psychophysiological effects of a web-based stress management system: A prospective, randomized controlled intervention study of IT and media workers [ISRCTN54254861] , 2005, BMC public health.

[78]  K. Søgaard,et al.  Effects on musculoskeletal pain, work ability and sickness absence in a 1-year randomised controlled trial among cleaners , 2011, BMC public health.

[79]  A. J. van der Beek,et al.  An Individually Based Lifestyle Intervention for Workers at Risk for Cardiovascular Disease: A Process Evaluation , 2011, American journal of health promotion : AJHP.

[80]  A. Stoddard,et al.  Tools for health: the efficacy of a tailored intervention targeted for construction laborers , 2007, Cancer Causes & Control.

[81]  Traci L Toomey,et al.  Worksite environment intervention to prevent obesity among metropolitan transit workers. , 2010, Preventive medicine.

[82]  Andrea Furlan,et al.  Updated Method Guidelines for Systematic Reviews in the Cochrane Collaboration Back Review Group , 2003, Spine.

[83]  A. Burdorf,et al.  The role of obesity and lifestyle behaviours in a productive workforce , 2010, Occupational and Environmental Medicine.

[84]  S. Ziebland,et al.  Analysing qualitative data , 2000, BMJ : British Medical Journal.

[85]  Kent Jacob Nielsen,et al.  Realistic evaluation as a new way to design and evaluate occupational safety interventions , 2012 .

[86]  T. Paulussen,et al.  Determinants of innovation within health care organizations: literature review and Delphi study. , 2004, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[87]  V. H. Hildebrandt,et al.  The design of a real-time formative evaluation of the implementation process of lifestyle interventions at two worksites using a 7-step strategy (BRAVO@Work) , 2012, BMC Public Health.

[88]  Bryan J Weiner,et al.  Using organization theory to understand the determinants of effective implementation of worksite health promotion programs. , 2008, Health education research.