Health systems performance assessment in low-income countries: learning from international experiences

BackgroundThe study aimed at developing a set of attributes for a ‘good’ health system performance assessment (HSPA) framework from literature and experiences in different contexts and using the attributes for a structured approach to lesson learning for low-income countries (LICs).MethodsLiterature review to identify relevant attributes for a HSPA framework; attribute validation for LICs in general, and for Uganda in particular, via a high-level Ugandan expert group; and, finally, review of a selection of existing HSPA frameworks using these attributes.ResultsLiterature review yielded six key attributes for a HSPA framework: an inclusive development process; its embedding in the health system’s conceptual model; its relation to the prevailing policy and organizational set-up and societal context; the presence of a concrete purpose, constitutive dimensions and indicators; an adequate institutional set-up; and, its capacity to provide mechanisms for eliciting change in the health system. The expert group contextualized these attributes and added one on the adaptability of the framework.Lessons learnt from the review of a selection of HSPA frameworks using the attributes include: it is possible and beneficial to involve a range of stakeholders during the process of development of a framework; it is important to make HSPA frameworks explicit; policy context can be effectively reflected in the framework; there are marked differences between the structure and content of frameworks in high-income countries, and low- and middle-income countries; champions can contribute to put HSPA high on the agenda; and mechanisms for eliciting change in the health system should be developed alongside the framework.ConclusionIt is possible for LICs to learn from literature and the experience of HSPA in other contexts, including HICs. In this study a structured approach to lesson learning included the development of a list of attributes for a ‘good’ HSPA framework. The attributes thus derived can be utilized by LICs like Uganda seeking to develop/adjust their HSPA frameworks as guidelines or a check list, while taking due consideration of the specific context. The review of frameworks from varied contexts, highlighted varied experiences which provide lessons for LICs.

[1]  Oliver Groene,et al.  The World Health Organization Performance Assessment Tool for Quality Improvement in Hospitals (PATH): an analysis of the pilot implementation in 37 hospitals. , 2008, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[2]  A. Gelijns,et al.  Perspective: Improving Health: The Reason Performance Measurement Matters , 1998 .

[3]  F. Scutchfield,et al.  Public health performance. , 2009, American journal of preventive medicine.

[4]  H. Davies,et al.  Payment for performance in health care , 2008, BMJ : British Medical Journal.

[5]  L. Nestman,et al.  The utilization of systematic outcome mapping to improve performance management in health care , 2006, Health services management research.

[6]  R. Brugha,et al.  A stakeholder approach towards hospital accreditation in India. , 2001, Health policy and planning.

[7]  N. Lustig Investing in Health for Economic Development : The Case of Mexico , 2006 .

[8]  Vicente Navarro,et al.  Assessment of the World Health Report 2000 , 2000, The Lancet.

[9]  A. Handler,et al.  A conceptual framework to measure performance of the public health system. , 2001, American journal of public health.

[10]  Craig Gibson,et al.  Assessment Essentials: Planning, Implementing, and Improving Assessment in Higher Education , 2000 .

[11]  M. Kruk,et al.  Assessing health system performance in developing countries: a review of the literature. , 2008, Health policy.

[12]  Onyebuchi A Arah,et al.  A conceptual framework for the OECD Health Care Quality Indicators Project. , 2006, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[13]  P. Smith,et al.  Measuring health system performance , 2002, The European Journal of Health Economics.

[14]  Conclusions , 1989 .

[15]  José Maria Paganni,et al.  Evaluating the quality of medical care. , 1966, The Milbank Memorial Fund quarterly.

[16]  A. Donabedian Evaluating the quality of medical care. 1966. , 1966, The Milbank quarterly.

[17]  A. Hyder Misunderstanding the World Health Report 2000. , 2002, American journal of public health.

[18]  C. Tashobya,et al.  THE EFFORT TO ACHIEVE THE MILLENIUM DEVELOPMENT GOALS IN UGANDA: REACHING FOR THE SKY? , 2004 .

[19]  Rifat Atun,et al.  Health systems, systems thinking and innovation. , 2012, Health policy and planning.

[20]  Irene Papanicolas,et al.  Performance measurement for health system improvement : experiences, challenges and prospects , 2010 .

[21]  Carla AbouZahr,et al.  From data to policy: good practices and cautionary tales , 2007, The Lancet.

[22]  A. Mills,et al.  Progress towards universal coverage: the health systems of Ghana, South Africa and Tanzania. , 2012, Health policy and planning.

[23]  A. Edward,et al.  Configuring Balanced Scorecards for Measuring Health System Performance: Evidence from 5 Years' Evaluation in Afghanistan , 2011, PLoS medicine.

[24]  J. Kirigia,et al.  The essence of governance in health development , 2011, International archives of medicine.

[25]  J. Ibrahim,et al.  Performance indicators from all perspectives. , 2001, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[26]  A. Williams,et al.  Science or marketing at WHO? A commentary on 'World Health 2000'. , 2001, Health economics.

[27]  Michael Grüninger,et al.  Introduction , 2002, CACM.

[28]  R. Kaplan,et al.  The balanced scorecard--measures that drive performance. , 2015, Harvard business review.

[29]  C. Murray,et al.  A framework for assessing the performance of health systems. , 2000, Bulletin of the World Health Organization.

[30]  Talcott Parsons,et al.  Social Systems and the Evolution of Action Theory@@@Action and the Human Condition , 1980 .

[31]  J. Braithwaite,et al.  Health care as a complex adaptive system , 2013 .

[32]  G R Baker,et al.  A Review of Organizational Performance Assessment in Health Care , 1998, Health services management research.

[33]  W. Chiu,et al.  Development and implementation of a nationwide health care quality indicator system in Taiwan. , 2007, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[34]  F Champagne,et al.  A performance assessment framework for hospitals: the WHO regional office for Europe PATH project. , 2005, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[35]  P. Barron,et al.  The District Health Barometer Year 1 , 2005 .

[36]  S. Leatherman,et al.  Does certification improve medical standards? , 2006, BMJ : British Medical Journal.

[37]  Tim Freeman,et al.  Using performance indicators to improve health care quality in the public sector: a review of the literature , 2002, Health services management research.

[38]  E. McGlynn,et al.  The outcomes utility index: will outcomes data tell us what we want to know? , 1998, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[39]  D M Eddy,et al.  Performance measurement: problems and solutions. , 1998, Health affairs.

[40]  V. Navarro The New Conventional Wisdom: An Evaluation of the Who Report Health Systems: Improving Performance , 2001, International journal of health services : planning, administration, evaluation.

[41]  Sania Nishtar,et al.  Framework for assessing governance of the health system in developing countries: gateway to good governance. , 2009, Health policy.

[42]  Erica Weir,et al.  Applying the balanced scorecard to local public health performance measurement: deliberations and decisions , 2009, BMC public health.

[43]  P. Shekelle Performance measurement for health system improvement: Public performance reporting on quality information , 2010 .

[44]  S. Leatherman,et al.  Improving performance using indicators. Recent experiences in the United States, the United Kingdom, and Australia. , 2001, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[45]  E. McGlynn Selecting Common Measures of Quality and System Performance , 2003, Medical care.

[46]  Claude Sicotte,et al.  Hospital performance: competing or shared values? , 2008, Health Policy.

[47]  Richard Rose,et al.  Lesson-drawing in Public Policy: A Guide to Learning Across Time and Space , 1993 .

[48]  P. Barron,et al.  Health information systems in South Africa , 2011 .

[49]  B. Starfield,et al.  World Health Report 2000: how it removes equity from the agenda for public health monitoring and policy. , 2001, BMJ : British Medical Journal.

[50]  Ulrich Rendtel,et al.  Editorial , 2014, Journal of basic microbiology.

[51]  Fadi El-Jardali,et al.  Design, implementation and scaling up of the balanced scorecard for hospitals in Lebanon: policy coherence and application lessons for low and middle income countries. , 2011, Health policy.

[52]  Jerod M Loeb,et al.  The current state of performance measurement in health care. , 2004, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[53]  A. Bierman,et al.  Performance measurement and equity , 2007, BMJ : British Medical Journal.

[54]  J. Sterman Learning from evidence in a complex world. , 2006, American journal of public health.

[55]  Sabrina T. Wong,et al.  Developing a Performance Measurement Framework and Indicators for Community Health Service Facilities in Urban China , 2010, BMC family practice.

[56]  Holly Jordan Lanham,et al.  Implications of complex adaptive systems theory for the design of research on health care organizations , 2009, Health care management review.

[57]  Anne Karin Lindahl,et al.  Paying for performance to improve the delivery of health interventions in low- and middle-income countries . , 2012, The Cochrane database of systematic reviews.

[58]  Jeremy Hurst,et al.  Performance measurement and performance management in OECD health systems , 2001 .

[59]  Scott C. Williams,et al.  The perceived impact of public reporting hospital performance data: interviews with hospital staff. , 2011, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[60]  G R Baker,et al.  A Conceptual Framework for the Analysis of Health Care Organizations' Performance , 1998, Health services management research.

[61]  A J ROURKE,et al.  Evaluating the quality of medical care. , 1957, Hospital progress.

[62]  B. Criel,et al.  The Health System Dynamics Framework: The introduction of an analytical model for health system analysis and its application to two case-studies , 2012 .

[63]  Onyebuchi A Arah,et al.  Comparing health system performance assessment and management approaches in the Netherlands and Ontario, Canada , 2007, BMC Health Services Research.

[64]  N. Weinberg Using performance measures to identify plans of action to improve care. , 2001, The Joint Commission journal on quality improvement.

[65]  O A Arah,et al.  Developing a national performance indicator framework for the Dutch health system. , 2004, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[66]  A. Mills,et al.  What can we learn from international comparisons of health systems and health system reform? , 2000, Bulletin of the World Health Organization.

[67]  O A Arah,et al.  Conceptual frameworks for health systems performance: a quest for effectiveness, quality, and improvement. , 2003, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[68]  Alan Williams Science or marketing at who? Rejoinder from Alan Williams , 2001 .

[69]  S. Leatherman,et al.  Public disclosure of health care performance reports: experience, evidence and issues for policy. , 1999, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[70]  P. Allotey,et al.  Health systems performance in sub-Saharan Africa: governance, outcome and equity , 2011, BMC public health.

[71]  H Labelle,et al.  A new perspective on the health of Canadians. , 1976, AARN news letter.

[72]  John Rohrbaugh,et al.  A Spatial Model of Effectiveness Criteria: Towards a Competing Values Approach to Organizational Analysis , 1983 .

[73]  G. Martin Globalization and Health , 2005 .

[74]  G. Westert,et al.  Correlates of health and healthcare performance: applying the Canadian health indicators framework at the provincial-territorial level , 2005, BMC health services research.

[75]  Reporting health care performance: learning from the past, prospects for the future. , 2002, Journal of evaluation in clinical practice.

[76]  David Grayson,et al.  Everybody's Business , 2001 .