Using routine health information systems for well-designed health evaluations in low- and middle-income countries.

Routine health information systems (RHISs) are in place in nearly every country and provide routinely collected full-coverage records on all levels of health system service delivery. However, these rich sources of data are regularly overlooked for evaluating causal effects of health programmes due to concerns regarding completeness, timeliness, representativeness and accuracy. Using Mozambique's national RHIS (Módulo Básico) as an illustrative example, we urge renewed attention to the use of RHIS data for health evaluations. Interventions to improve data quality exist and have been tested in low-and middle-income countries (LMICs). Intrinsic features of RHIS data (numerous repeated observations over extended periods of time, full coverage of health facilities, and numerous real-time indicators of service coverage and utilization) provide for very robust quasi-experimental designs, such as controlled interrupted time-series (cITS), which are not possible with intermittent community sample surveys. In addition, cITS analyses are well suited for continuously evolving development contexts in LMICs by: (1) allowing for measurement and controlling for trends and other patterns before, during and after intervention implementation; (2) facilitating the use of numerous simultaneous control groups and non-equivalent dependent variables at multiple nested levels to increase validity and strength of causal inference; and (3) allowing the integration of continuous 'effective dose received' implementation measures. With expanded use of RHIS data for the evaluation of health programmes, investments in data systems, health worker interest in and utilization of RHIS data, as well as data quality will further increase over time. Because RHIS data are ministry-owned and operated, relying upon these data will contribute to sustainable national capacity over time.

[1]  S. Iliffe,et al.  Bmc Medical Research Methodology Open Access the Hawthorne Effect: a Randomised, Controlled Trial , 2022 .

[2]  P. Rothwell,et al.  External validity of randomised controlled trials: “To whom do the results of this trial apply?” , 2005, The Lancet.

[3]  J. Bryce,et al.  Measuring Coverage in MNCH: Evaluation of Community-Based Treatment of Childhood Illnesses through Household Surveys , 2013, PLoS medicine.

[4]  A K Wagner,et al.  Segmented regression analysis of interrupted time series studies in medication use research , 2002, Journal of clinical pharmacy and therapeutics.

[5]  H Goldstein,et al.  Multilevel time series models with applications to repeated measures data. , 1994, Statistics in medicine.

[6]  A. Wagenaar,et al.  Effects of alcohol tax increases on alcohol-related disease mortality in Alaska: time-series analyses from 1976 to 2004. , 2009, American journal of public health.

[7]  A. Rowe Potential of integrated continuous surveys and quality management to support monitoring, evaluation, and the scale-up of health interventions in developing countries. , 2009, The American journal of tropical medicine and hygiene.

[8]  Kenneth Sherr,et al.  Effects of a health information system data quality intervention on concordance in Mozambique: time-series analyses from 2009–2012 , 2015, Population Health Metrics.

[9]  Yoonjoung Choi,et al.  A systematic review of Demographic and Health Surveys: data availability and utilization for research. , 2012, Bulletin of the World Health Organization.

[10]  J. Bryce,et al.  Evidence-based public health: moving beyond randomized trials. , 2004, American journal of public health.

[11]  Alexander C. Wagenaar,et al.  Mandatory seat belt laws in eight states: a time-series evaluation☆ , 1988 .

[12]  P. O'Malley,et al.  Effects of minimum drinking age laws on alcohol use, related behaviors and traffic crash involvement among American youth: 1976-1987. , 1991, Journal of studies on alcohol.

[13]  P. Byass The Imperfect World of Global Health Estimates , 2010, PLoS medicine.

[14]  K. Mate,et al.  Challenges for Routine Health System Data Management in a Large Public Programme to Prevent Mother-to-Child HIV Transmission in South Africa , 2009, PloS one.

[15]  C. AbouZahr,et al.  Health information systems: the foundations of public health. , 2005, Bulletin of the World Health Organization.

[16]  S. Gimbel,et al.  Strengthening integrated primary health care in Sofala, Mozambique , 2013, BMC Health Services Research.

[17]  I. Kleinschmidt,et al.  Impact assessment of malaria vector control using routine surveillance data in Zambia: implications for monitoring and evaluation , 2012, Malaria Journal.

[18]  J. Pfeiffer International NGOs and primary health care in Mozambique: the need for a new model of collaboration. , 2003, Social science & medicine.

[19]  James Price,et al.  The impact of a smoking ban on hospital admissions for coronary heart disease. , 2007, Preventive medicine.

[20]  K. Hofman,et al.  Impact of supplemental immunisation activity (SIA) campaigns on health systems: findings from South Africa , 2013, Journal of Epidemiology & Community Health.

[21]  S. P. Kachur,et al.  Caution is required when using health facility-based data to evaluate the health impact of malaria control efforts in Africa , 2009, Malaria Journal.

[22]  Christopher Zorn Generalized Estimating Equation Models for Correlated Data: A Review with Applications , 2001 .

[23]  Stephen S Lim,et al.  Tracking progress towards universal childhood immunisation and the impact of global initiatives: a systematic analysis of three-dose diphtheria, tetanus, and pertussis immunisation coverage , 2008, The Lancet.

[24]  W. Shadish,et al.  Experimental and Quasi-Experimental Designs for Generalized Causal Inference , 2001 .

[25]  G. Glass,et al.  Design and analysis of time-series experiments , 1975 .

[26]  J. Bryce,et al.  Monitoring child survival in ‘real time’ using routine health facility records: results from Malawi , 2013, Tropical medicine & international health : TM & IH.

[27]  Kenneth Sherr,et al.  An assessment of routine primary care health information system data quality in Sofala Province, Mozambique , 2011, Population health metrics.

[28]  M. Lynch,et al.  Viewpoint: Evaluating the impact of malaria control efforts on mortality in sub‐Saharan Africa , 2007, Tropical medicine & international health : TM & IH.

[29]  Linda C. Li,et al.  Recruitment and retention of occupational therapists and physiotherapists in rural regions: a meta-synthesis , 2013, BMC Health Services Research.

[30]  A. Blanc Demographic and health surveys , 1991 .

[31]  C. Amoroso,et al.  Improving health information systems for decision making across five sub-Saharan African countries: Implementation strategies from the African Health Initiative , 2013, BMC Health Services Research.

[32]  Jørn Braa,et al.  Improving quality and use of data through data-use workshops: Zanzibar, United Republic of Tanzania. , 2012, Bulletin of the World Health Organization.

[33]  M. Aanestad,et al.  Analysing the quality of routine malaria data in Mozambique , 2004, Malaria Journal.

[34]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1972 .

[35]  Robert E Black,et al.  Measuring impact in the Millennium Development Goal era and beyond: a new approach to large-scale effectiveness evaluations , 2011, The Lancet.

[36]  Kenneth Sherr,et al.  An assessment of the accuracy and availability of data in electronic patient tracking systems for patients receiving HIV treatment in central Mozambique , 2012, BMC Health Services Research.

[37]  K. Mate,et al.  Improving public health information: a data quality intervention in KwaZulu-Natal, South Africa. , 2012, Bulletin of the World Health Organization.

[38]  J. Lloyd,et al.  The immunization data quality audit: verifying the quality and consistency of immunization monitoring systems. , 2005, Bulletin of the World Health Organization.

[39]  Richard A. Berk,et al.  Applied Time Series Analysis for the Social Sciences , 1980 .

[40]  B. Wagenaar,et al.  Effects of health-system strengthening on under-5, infant, and neonatal mortality: 11-year provincial-level time-series analyses in Mozambique , 2014, The Lancet. Global health.

[41]  Richard J. K. Taylor,et al.  The decline in Australian young male suicide. , 2007, Social science & medicine.

[42]  A. Wagenaar,et al.  Public Health Law Research: Theory and Methods , 2014 .

[43]  Alexander C. Wagenaar,et al.  The Value of Interrupted Time-Series Experiments for Community Intervention Research , 2000, Prevention Science.

[44]  Joanna Schellenberg,et al.  Measuring implementation strength: Literature review draft report 2012 , 2012 .

[45]  R. Cibulskis,et al.  Information systems for health sector monitoring in Papua New Guinea. , 2002, Bulletin of the World Health Organization.

[46]  O. Ronveaux,et al.  Accuracy and quality of immunization information systems in forty‐one low income countries , 2009, Tropical medicine & international health : TM & IH.