Methodology

The methodology for this report was designed to generate new and actionable insights on reducing under-five mortality (U5M). More specifically, the research process was designed to evaluate both the selection of interventions against U5M; the various means of executing those interventions; and the contextual factors that abetted or hindered the reduction of U5M.

Project Framework

The identification of evidence-based interventions (EBIs) and policies that exemplar nations like Ethiopia have used to reduce U5M is fairly straightforward. However, the identification of actionable lessons is a more complex matter.

Truly useful guidance on how exemplar nations selected, adapted, implemented, and sustained their EBIs are often missing from both published materials and documents produced outside of formal commercial or academic channels (so-called “gray literature”).

To overcome this, the research team designed an implementation-science framework specifically for this project. Because policies and interventions brought different results in different countries, implementation science offers important tools for how to think more holistically about how and why countries were able to reduce U5M – and about the lessons that policymakers might usefully draw from these outcomes.

This approach combines elements of existing frameworks, notably Aarons et al’s Exploration, Preparation, Implementation, and Sustainment (EPIS) from 2011; and Proctor et al’s implementation outcomes (Feasibility, Fidelity, Acceptability, Reach, and Effectiveness) from 2010. We also added a new step – Adaptation – to the EPIS framework (resulting in a modified abbreviation – EPIAS).¹

Desk Review

The Strategic Analysis, Research, & Training (START) Center at the University of Washington undertook an extensive review of available information and published data on the rates and progress of U5M, including policies, strategies, EBIs available to potential exemplar countries, and the uptake and implementation of these EBIs in Ethiopia.

Initial secondary research was performed through MEDLINE (PubMed) and Google Scholar using the search terms “child mortality” or “under-5 mortality.” Further searches included specific EBIs, causes of death, or contextual factors as search terms (e.g. “insecticide-treated nets,” “malaria,” or “community health workers”).

Initial desk research by the START Center was synthesized and then reviewed by the UGHE team for accuracy and completeness. Following this, UGHE provided additional support to increase the capture of published literature relevant to the work.

The desk review was an iterative process, with ongoing additions occurring throughout the primary-research process as additional sources (published articles, reports, case studies) were identified. 

Primary Research

In collaboration with our in-country partners in Ethiopia (MERQ Consultancy), we identified key informants reflecting a broad range of experience and viewpoints. These interviewees were chosen based on the topics identified in the desk review, and through other analyses in close collaboration with in-country partners. We placed a particularly high priority on those interviewees who could provide information on the EPIAS stages during the period of study.

Key informants included current and former Federal Ministry of Health employees responsible for high-level strategic direction either of the overall ministry or of specific disease or intervention areas. Other interviewees included implementing partners; representatives of multilateral organizations; and donor-group officials who had managed partner-supported or partner-led activities.

Some of these interviewees represented more than one area or role based on their experience over the 16 years of the study period, and provided information from each of these multiple viewpoints. While we sought out individuals who had been active during the study period, the interviews also captured highly relevant contextual information from before 2000 and after 2016.

The interviews were designed to elicit contextual factors at the relevant global, national, ministry, and local levels, and to identify additional sources of data and information which could be added to the knowledge base already developed from the desk review.

All interviews were led by the project research associates or the in-country principal investigator, with support from in-country researchers taking notes and operating recorders. Following the close of the interviews, notes were combined and the tape recordings (if allowed) were used to clarify responses as needed. All interviews were conducted in English.

Analysis and Synthesis

The UGHE team used a mixed-methods explanatory approach, applying the framework to understand the progress (or lack thereof) on each cause of death; the coverage levels of specific EBIs; and contextual conditions at the local, national, and global levels.

These analyses were further informed by the extensive work completed by other initiatives, including Countdown 2015, WHO maternal and child health initiatives, the International Center for Equity in Health, and others.

Interviews of key informants were coded by the researchers, and the research framework was used to extract the EPIAS steps and contextual factors. A priori codes for contextual factors were adapted and expanded as emerging themes were identified. Due to resource constraints and the range and diversity of interviewees, qualitative analysis using software was not planned.

The study was reviewed by the relevant institutional review boards in Rwanda and Ethiopia. All interviewees provided informed consent before interviews were conducted.

In addition, the research team collaborated with IHME to look at quantitative modeling results using a decomposition method, and also collaborated with the Johns Hopkins Bloomberg School of Public Health to model results using the Lives Saved Tool (LiST). These quantitative analyses complement the primary research by looking at what the models suggest about the likely contribution of specific interventions in reducing child mortality.

The decomposition analysis conducted by IHME breaks down changes over time for a series of factors that directly influence child mortality levels using the Das Gupta method. The overall change in mortality between years is divided into contributions from:

• Interventions and risk factors: Interventions and risk factors influence mortality rates through changes in the proportion of the population exposed to each, and through changes in their corresponding relative risks of mortality. Increased coverage of specific interventions is known to reduce mortality rates, whereas increased exposure to certain risk factors increases mortality rates. The relative risk for each specific disease outcome is established through a literature review.
• Population changes: The total number of deaths in a given year is a product of both age-specific mortality rates and the population size in each age group, so changes in both population growth and population age structure are factored into the decomposition. One example of the effect of population changes is if mortality rates are cut in half while the population size doubles in each age group, total deaths remain the same.

The Lives Saved Tool (LiST), developed by the Johns Hopkins Bloomberg School of Public Health, calculates changes in cause-specific mortality based on intervention coverage change, intervention effectiveness for that cause, and the percentage of cause-specific mortality sensitive to that intervention. Coverage data come from large-scale household surveys – typically Demographic and Health Surveys (DHS) and Multiple Indicator Cluster Surveys (MICS), as well as WHO/UNICEF and the WHO/UNICEF Joint Monitoring Programme for Water Supply, Sanitation and Hygiene (JMP).

Default effectiveness values come from systematic reviews, meta-analyses, Delphi estimations, and randomized control trials based upon the Child Health Epidemiology Reference Group guidelines. Baseline mortality is drawn from country-level estimates from DHS, WHO, UNICEF, UNFPA, World Bank Group, and the United Nations Population Division and the UN Inter-Agency Group for Child Mortality Estimation (IGME). Additionally, users who have more recent or alternative data sources can easily replace default data with their own.