Getting the most from after action reviews to improve global health security

BackgroundAfter Action Reviews (AARs) provide a means to observe how well preparedness systems perform in real world conditions and can help to identify – and address – gaps in national and global public health emergency preparedness (PHEP) systems. WHO has recently published guidance for voluntary AARs. This analysis builds on this guidance by reviewing evidence on the effectiveness of AARs as tools for system improvement and by summarizing some key lessons about ensuring that AARs result in meaningful learning from experience.ResultsEmpirical evidence from a variety of fields suggests that AARs hold considerable promise as tools of system improvement for PHEP. Our review of the literature and practical experience demonstrates that AARs are most likely to result in meaningful learning if they focus on incidents that are selected for their learning value, involve an appropriately broad range of perspectives, are conducted with appropriate time for reflection, employ systems frameworks and rigorous tools such as facilitated lookbacks and root cause analysis, and strike a balance between attention to incident specifics vs. generalizable capacities and capabilities.ConclusionsEmploying these practices requires a PHEP system that facilitates the preparation of insightful AARs, and more generally rewards learning. The barriers to AARs fall into two categories: concerns about the cultural sensitivity and context, liability, the political response, and national security; and constraints on staff time and the lack of experience and the requisite analytical skills. Ensuring that AARs fulfill their promise as tools of system improvement will require ongoing investment and a change in mindset. The first step should be to clarify that the goal of AARs is organizational learning, not placing blame or punishing poor performance. Based on experience in other fields, the buy-in of agency and political leadership is critical in this regard. National public health systems also need support in the form of toolkits, guides, and training, as well as research on AAR methods. An AAR registry could support organizational improvement through careful post-event analysis of systems’ own events, facilitate identification and sharing of best practices across jurisdictions, and enable cross-case analyses.

[1]  M. Stoto,et al.  Learning about after action reporting from the 2009 H1N1 pandemic: a workshop summary. , 2013, Journal of public health management and practice : JPHMP.

[2]  J. Carroll Organizational Learning Activities in High‐hazard Industries: The Logics Underlying Self‐Analysis , 1998 .

[3]  L. Gilson,et al.  Building the Field of Health Policy and Systems Research: Social Science Matters , 2011, PLoS medicine.

[4]  E. G. Lyman,et al.  NASA aviation safety reporting system , 1976 .

[5]  Paul D. Biddinger,et al.  Use of After Action Reports (AARs) to Promote Organizational and Systems Learning in Emergency Preparedness , 2012, International journal of environmental research and public health.

[6]  Hilde van der Togt,et al.  Publisher's Note , 2003, J. Netw. Comput. Appl..

[7]  Michael A. Stoto,et al.  Did Advances in Global Surveillance and Notification Systems Make a Difference in the 2009 H1N1 Pandemic?–A Retrospective Analysis , 2013, PloS one.

[8]  P. Biddinger,et al.  Be prepared--the Boston Marathon and mass-casualty events. , 2013, The New England journal of medicine.

[9]  John E. Morrison,et al.  Foundations of the After Action Review Process , 1999 .

[10]  Joshua Lederberg,et al.  Microbial Threats to Health: Emergence, Detection, and Response , 2003 .

[11]  J. Brown,et al.  Organizational Learning and Communities-of-Practice: Toward a Unified View of Working, Learning, and Innovation , 1991 .

[12]  M. Stoto,et al.  A public health emergency preparedness critical incident registry. , 2014, Biosecurity and bioterrorism : biodefense strategy, practice, and science.

[13]  E. Sorrell,et al.  Implementation of the International Health Regulations (2005) Through Cooperative Bioengagement , 2015, Front. Public Health.

[14]  Victor J. Friedman,et al.  A Multifacet Model of Organizational Learning , 2002 .

[15]  P. Pronovost,et al.  Effectiveness and efficiency of root cause analysis in medicine. , 2008, JAMA.

[16]  Joseph A. Allen,et al.  After-action reviews: a venue for the promotion of safety climate. , 2010, Accident; analysis and prevention.

[17]  Margaret Salter,et al.  After Action Reviews: Current Observations and Recommendations , 2007 .

[18]  Richard Sexton,et al.  A COMPARATIVE STUDY OF AFTER ACTION REVIEW (AAR) IN THE CONTEXT OF THE SOUTHERN AFRICA CRISIS , 2003 .

[19]  M. Stoto,et al.  Root-Cause Analysis for Enhancing Public Health Emergency Preparedness: A Brief Report of a Salmonella Outbreak in the Alamosa, Colorado, Water Supply , 2018, Journal of Public Health Management and Practice.

[20]  C. Dolea,et al.  World Health Organization , 1949, International Organization.

[21]  Julia E. Aledort,et al.  Facilitated Look-Backs: A New Quality Improvement Tool for Management of Routine Annual and Pandemic Influenza , 2006 .

[22]  G. Rodier,et al.  Joint external evaluation process: bringing multiple sectors together for global health security. , 2017, The Lancet. Global health.

[23]  Daniel L. Stufflebeam,et al.  Stufflebeam’s Improvement-Oriented Evaluation , 1985 .

[24]  Jonathan E. Suk,et al.  Enhancing Reporting of After Action Reviews of Public Health Emergencies to Strengthen Preparedness: A Literature Review and Methodology Appraisal , 2018, Disaster Medicine and Public Health Preparedness.

[25]  Joseph A. Allen,et al.  Implementing after-action review systems in organizations: Key principles and practical considerations , 2015 .

[26]  C. Quinn,et al.  Using "Mystery Patient" Drills to Assess Hospital Ebola Preparedness in New York City, 2014-2015. , 2017, Health security.

[27]  David A. Garvin,et al.  Learning in Action: A Guide to Putting the Learning Organization to Work , 2000 .

[28]  M. Stoto,et al.  A Public Health Preparedness Logic Model: Assessing Preparedness for Cross-border Threats in the European Region , 2017, Health security.

[29]  Scott I. Tannenbaum,et al.  Do Team and Individual Debriefs Enhance Performance? A Meta-Analysis , 2013, Hum. Factors.

[30]  NelsonChristopher,et al.  A Public Health Preparedness Logic Model: Assessing Preparedness for Cross-border Threats in the European Region , 2017 .

[31]  Dean F. Sittig,et al.  Ebola US Patient Zero: lessons on misdiagnosis and effective use of electronic health records , 2014, Diagnosis.

[32]  Shmuel Ellis,et al.  After-event reviews: drawing lessons from successful and failed experience. , 2005, The Journal of applied psychology.

[33]  R. Lipshitz,et al.  How Organizations Learn: Post-flight Reviews in an F-16 Fighter Squadron , 2006 .

[34]  Michael W. Morris,et al.  The Lessons We (Don't) Learn: Counterfactual Thinking and Organizational Accountability after a Close Call , 2000 .