Public Health Emergency Preparedness System Evaluation Criteria and Performance Metrics: A Review of Contributions of the CDC-Funded Preparedness and Emergency Response Research Centers

OBJECTIVES The US Centers for Disease Control and Prevention (CDC)-funded Preparedness and Emergency Response Research Centers (PERRCs) conducted research from 2008 to 2015 aimed to improve the complex public health emergency preparedness and response (PHEPR) system. This paper summarizes PERRC studies that addressed the development and assessment of criteria for evaluating PHEPR and metrics for measuring their efficiency and effectiveness. METHODS We reviewed 171 PERRC publications indexed in PubMed between 2009 and 2016. These publications derived from 34 PERRC research projects. We identified publications that addressed the development or assessment of criteria and metrics pertaining to PHEPR systems and describe the evaluation methods used and tools developed, the system domains evaluated, and the metrics developed or assessed. RESULTS We identified 29 publications from 12 of the 34 PERRC projects that addressed PHEPR system evaluation criteria and metrics. We grouped each study into 1 of 3 system domains, based on the metrics developed or assessed: (1) organizational characteristics (n = 9), (2) emergency response performance (n = 12), and (3) workforce capacity or capability (n = 8). These studies addressed PHEPR system activities including responses to the 2009 H1N1 pandemic and the 2011 tsunami, as well as emergency exercise performance, situational awareness, and workforce willingness to respond. Both PHEPR system process and outcome metrics were developed or assessed by PERRC studies. CONCLUSIONS PERRC researchers developed and evaluated a range of PHEPR system evaluation criteria and metrics that should be considered by system partners interested in assessing the efficiency and effectiveness of their activities. Nonetheless, the monitoring and measurement problem in PHEPR is far from solved. Lack of standard measures that are readily obtained or computed at local levels remains a challenge for the public health preparedness field. (Disaster Med Public Health Preparedness. 2018;page 1 of 13).

[1]  J. Links,et al.  EPPM and Willingness to Respond: The Role of Risk and Efficacy Communication in Strengthening Public Health Emergency Response Systems , 2014, Health communication.

[2]  Christina Catlett,et al.  Assessment of psychological preparedness and emergency response willingness of local public health department and hospital workers. , 2012, International journal of emergency mental health.

[3]  N. Lurie,et al.  Assessing public health emergency preparedness: concepts, tools, and challenges. , 2007, Annual review of public health.

[4]  Mary V. Davis,et al.  Effects of Performance Improvement Programs on Preparedness Capacities , 2014, Public health reports.

[5]  Mary V. Davis,et al.  Declining trends in local health department preparedness capacities. , 2014, American journal of public health.

[6]  Luis Duran,et al.  Developing a measure of local agency adaptation to emergencies: a metric. , 2012, Evaluation and program planning.

[7]  M. Stoto,et al.  A Review of Instruments Assessing Public Health Preparedness , 2005, Public health reports.

[8]  P. Biddinger,et al.  A Conceptual Framework to Measure Systems’ Performance during Emergency Preparedness Exercises , 2014, International journal of environmental research and public health.

[9]  Carolyn J Leep,et al.  2005 National profile of local health departments. , 2006, Journal of public health management and practice : JPHMP.

[10]  Alan Kirschenbaum,et al.  Measuring the Effectiveness of Disaster Management Organizations , 2004, International Journal of Mass Emergencies & Disasters.

[12]  Michael A. Stoto,et al.  Did Advances in Global Surveillance and Notification Systems Make a Difference in the 2009 H1N1 Pandemic?–A Retrospective Analysis , 2013, PloS one.

[13]  M. Stoto,et al.  A Peer Assessment Approach for Learning from Public Health Emergencies , 2014, Public health reports.

[14]  Dorothy Bernard,et al.  Development of an Online Toolkit for Measuring Performance in Health Emergency Response Exercises , 2015, Prehospital and Disaster Medicine.

[15]  A. Rodday,et al.  Public health emergency preparedness at the local level: results of a national survey. , 2009, Health services research.

[16]  Matthew Biggerstaff,et al.  Epidemiology of 2009 pandemic influenza A (H1N1) in the United States. , 2011, Clinical infectious diseases : an official publication of the Infectious Diseases Society of America.

[17]  Jennifer C. Hunter,et al.  Integrating a framework for conducting public health systems research into statewide operations-based exercises to improve emergency preparedness , 2012, BMC Public Health.

[18]  M. Stoto,et al.  Learning from successful school-based vaccination clinics during 2009 pH1N1. , 2014, The Journal of school health.

[19]  Lisle Hites,et al.  Lessons about the state and local public health system response to the 2009 H1N1 pandemic: a workshop summary. , 2013, Journal of public health management and practice : JPHMP.

[20]  K. Witte.  Putting the fear back into fear appeals: The extended parallel process model , 1992 .

[21]  B. Altevogt,et al.  Research Priorities in Emergency Preparedness and Response for Public Health Systems: A Letter Report , 2008 .

[22]  Jane A. Kushma,et al.  Preparedness and Emergency Response Research Centers: Early Returns on Investment in Evidence-Based Public Health Systems Research , 2014, Public health reports.

[23]  M. Stoto.  Measuring and assessing public health emergency preparedness. , 2013, Journal of public health management and practice : JPHMP.

[24]  Mary V. Davis,et al.  Improving Public Health Preparedness Capacity Measurement: Development of the Local Health Department Preparedness Capacities Assessment Survey , 2013, Disaster Medicine and Public Health Preparedness.

[25]  M. Stoto,et al.  Local health department public vaccination clinic success during 2009 pH1N1. , 2013, Journal of public health management and practice : JPHMP.

[26]  Ran D Balicer,et al.  Determinants of emergency response willingness in the local public health workforce by jurisdictional and scenario patterns: a cross-sectional survey , 2012, BMC Public Health.

[27]  A. Afifi,et al.  Reliability and Validity of the Assessment for Disaster Engagement with Partners Tool (ADEPT) for Local Health Departments , 2014, Public health reports.

[28]  M. Stoto,et al.  A Toolkit to Assess Medical Reserve Corps Units' Performance , 2010, Disaster Medicine and Public Health Preparedness.

[29]  Jonathan M. Links,et al.  Assessment of Local Public Health Workers' Willingness to Respond to Pandemic Influenza through Application of the Extended Parallel Process Model , 2009, PloS one.

[30]  M. Hunter,et al.  Public Health System Response to Extreme Weather Events. , 2016, Journal of public health management and practice : JPHMP.

[31]  Piltch-LoebRachael,et al.  A Public Health Emergency Preparedness Critical Incident Registry , 2014 .

[32]  Jennifer C. Hunter,et al.  Public Health Response Systems In-Action: Learning from Local Health Departments’ Experiences with Acute and Emergency Incidents , 2013, PloS one.

[33]  Shoukat H Qari,et al.  Preparedness and Emergency Response Research Centers: Using a Public Health Systems Approach to Improve All-Hazards Preparedness and Response , 2014, Public health reports.

[34]  Paul D. Biddinger,et al.  Use of After Action Reports (AARs) to Promote Organizational and Systems Learning in Emergency Preparedness , 2012, International journal of environmental research and public health.

[35]  Michael A. Stoto,et al.  The Effectiveness of U.S. Public Health Surveillance Systems for Situational Awareness during the 2009 H1N1 Pandemic: A Retrospective Analysis , 2012, PloS one.

[36]  Jennifer C. Hunter,et al.  Local Public Health System Response to the Tsunami Threat in Coastal California following the Tōhoku Earthquake , 2012, PLoS currents.

[37]  Margaret A Potter,et al.  The adaptive response metric: toward an all-hazards tool for planning, decision support, and after-action analytics. , 2013, Journal of public health management and practice : JPHMP.

[38]  Mary V. Davis,et al.  Temporal Trends in Local Public Health Preparedness Capacity , 2014 .