Methodology

The Rwanda Under-Five Mortality (U5M) study is a collaboration between The University of Global Health Equity (UGHE), the Bill & Melinda Gates Foundation, and Gates Ventures, as part of the Exemplars in Global Health project.

This has two broad aims: First, developing and testing an implementation framework and mixed-methods approach to understand the success of exemplar countries; second, identifying implementation strategies and contextual factors that other nations could adapt and adopt to reduce U5M within their own borders.

Project Framework

The UGHE and Gates Ventures teams proceeded with primary and desk research, informed by an implementation-science framework designed specifically for this project.

Because the same policies and interventions brought different results in different countries, implementation science offers important tools for how to think more holistically about how and why countries were able to reduce U5M, and from where lessons in replication can be drawn.

The Rwanda U5M project’s initial work included identifying evidence-based interventions (EBIs) that had been found to directly reduce U5M, dividing the interventions between those targeting the neonatal period (birth to 28 days) and post-neonatal early childhood (28 days to 4 years). For a full listing of the EBIs considered, see the tables at the end of this document.

While we are often able to identify policies and evidence-based interventions chosen by a country to reduce U5M, the key lessons in how these were chosen, adapted, implemented, and sustained are often missing from available published or gray literature.

To advance our research, we developed a framework to understand the contribution of contextual factors and the different levels of actors involved: global, national, ministry, subnational, facility, and community.

That framework emerged from an exhaustive consideration of current models, and incorporates elements from several of them, along with vital insights from primary investigators on approaches to the interpretation of evidence and the design of primary-research tools.

The primary frameworks and implementation science resources we drew from include:

Exploration, Preparation, Implementation, Sustainment (EPIS):¹

This framework walks through four key steps of the implementation process needed to achieve long-term change: exploration; adoption decision/preparation; active implementation; and sustainment.

Re-AIM:²

This evaluation framework breaks down implementation outcomes into the four elements of its acronym: reach (coverage); effectiveness; adoption (range and proportion of individuals and organizations willing to participate); implementation (fidelity, time, cost and adaptations made); and maintenance (institutionalization into routine care and policies and long-term impact). It is designed to demonstrate the range of factors that influence success or failure at the individual and broader levels.

Consolidated Framework for Implementation Research (CFIR):³

This framework serves as a guide to understand the contextual factors that influenced the success or failure of implementation of a specific intervention. These include the outer context; the inner (organizational) context; the characteristics of the intervention; the implementation approach; and the individual actors responsible for implementation.

Implementation Outcomes (Proctor et al):⁴

This approach distinguishes implementation outcomes from the more traditionally measured intervention and system outcomes.

None of the above frameworks could, by themselves, account for the full range of implementation strategies that Rwanda and its partners undertook at the national, subnational, and care-delivery levels.

That said, the final framework certainly incorporated elements from established approaches. This included the choice of and implementation of EBIs (drawing from EPIS), as well as facilitators and barriers at the local, national and global levels (drawing from CFIR).

By combining all of these frameworks, and adding some new elements, we developed an approach that could guide us in prioritizing areas for primary research, interpreting the secondary research, and synthesizing our findings. A graphical illustration of the team’s final framework may be found at the end of this document.

Desk Review

The UGHE/Gates Ventures team undertook an extensive review of available information and published data on the current state of U5M in Rwanda and worldwide. It assessed the policies, strategies, and specific evidence-based interventions available to potential exemplar countries, as well as the implementation of these EBIs in five exemplar countries, including Rwanda. All five nations had succeeded in reducing U5M mortality beyond those experienced by other nations in the same region or at similar levels of wealth.

The team conducted its initial secondary research through MEDLINE (PubMed) and Google Scholar, using the search terms “child mortality” or “under-5 mortality” and the country’s name. Further searches included specific EBIs, causes of death, or contextual factors as search terms (e.g. “insecticide-treated nets,” “malaria,” or “community health workers”).

The UGHE team reviewed this initial secondary research for accuracy and completeness. This desk review was an iterative process, with ongoing additions throughout the concurrent primary-research process as additional sources (published articles, reports, case studies) were identified.

The team purposely did not include in-depth reviews of important broad interventions that contributed to U5M reduction - including education, poverty reduction, improved sanitation, and nutrition programs - but did include some evidence of increased coverage where available.

Primary Research

Informed by the framework and review of relevant literature on contextual factors and implementation outcomes, the team organized primary-research interviews with key informants who had direct experience in supervising or carrying out Rwanda’s U5M interventions during the time period covered in this report.

The purpose of these interviews was to gain an explanatory evaluation of the results from the desk review, identify additional areas of research, and fill identified gaps in understanding.

The team developed interview templates for three categories of informants: Ministry of Health officials and staff; project managers and implementers for specific causes of death or EBIs; and other partners at the national or global level.

The informants included current and former MOH employees, along with representatives of non-government organizations (NGOs); institutional donors, and multinational agencies. We focused on individuals active in the time period between 2000 and 2015, but were able to also capture some experiences from the formative post-genocide period of 1995-2000 as well as insights on developments since 2015.

Key informants were chosen based on the topics identified in the desk review and the close collaboration of in-country collaborators. In selecting interviewees, the research time placed a priority on finding individuals who could shed light on those EBIs which were reported as most successful, as well as on those for which no evidence of implementation could be found.

The interviews themselves were designed to address the entire intervention-implementation process, from conception through sustainment.

Interviewees were informed about the goals and structure of the project, and consent for participation and recording was obtained separately from the interview. The sole purpose of recording was to review the accuracy of notes.

Analysis and Synthesis

The UGHE team used a mixed-methods explanatory approach, applying the framework to understand the progress (or lack thereof) for each cause of death and the coverage of chosen evidence-based interventions, as well as facilitators and barriers at the local, national, and global levels.

This approach was designed to understand how and why Peru was able to achieve success in decreasing U5M, and the primary obstacles it faced. The analyses were also informed by other projects, including Countdown 2015, equity analyses from the International Center for Equity in Health, and geospatial mapping from IHME, among others.

In addition, the research team collaborated with IHME to look at quantitative modeling results using a decomposition method, and also collaborated with the Johns Hopkins Bloomberg School of Public Health to model results using the Lives Saved Tool (LiST).These quantitative analyses complement the primary research by looking at what the models suggest about the likely contribution of specific interventions in reducing child mortality.

The decomposition analysis conducted by IHME breaks down changes over time for a series of factors that directly influence child mortality levels using the Das Gupta method. The overall change in mortality between years is divided into contributions from:

• Interventions and risk factors: Interventions and risk factors influence mortality rates through changes in the proportion of the population exposed to each, and through changes in their corresponding relative risks of mortality. Increased coverage of specific interventions is known to reduce mortality rates, whereas increased exposure to certain risk factors increases mortality rates. The relative risk for each specific disease outcome is established through a literature review.
• Population changes: The total number of deaths in a given year is a product of both age-specific mortality rates and the population size in each age group, so changes in both population growth and population age structure are factored into the decomposition. One example of the effect of population changes is if mortality rates are cut in half while the population size doubles in each age group, total deaths remain the same.

The Lives Saved Tool (LiST), developed by the Johns Hopkins Bloomberg School of Public Health, calculates changes in cause-specific mortality based on intervention coverage change, intervention effectiveness for that cause, and the percentage of cause-specific mortality sensitive to that intervention. Coverage data come from large-scale household surveys – typically Demographic and Health Surveys (DHS) and Multiple Indicator Cluster Surveys (MICS), as well as WHO/UNICEF and the WHO/UNICEF Joint Monitoring Programme for Water Supply, Sanitation and Hygiene (JMP).

Default effectiveness values come from systematic reviews, meta-analyses, Delphi estimations, and randomized control trials based upon the Child Health Epidemiology Reference Group guidelines. Baseline mortality is drawn from country-level estimates from DHS, WHO, UNICEF, UNFPA, World Bank Group, and the United Nations Population Division and the UN Inter-Agency Group for Child Mortality Estimation (IGME). Additionally, users who have more recent or alternative data sources can easily replace default data with their own.

Human Subjects Review

The UGHE/Gates Ventures team’s work was approved by - and conducted with the support of - the Ministry of Health (MOH), the National Health Research Committee, and the Rwanda Biomedical Center’s Division of Maternal, Child and Community Health. A waiver was obtained from the Rwanda National Ethics Committee on the grounds that this research was to be conducted entirely retrospectively, using de-identified data and desk reviews of existing reports, with informed consent obtained from all interview participants.

No quotes or specific viewpoints identifiable to the source have been included without explicit permission. All recordings and interviews were kept in password-protected computers, and stored on a limited-access Google Drive with all identifications of interviewees removed. All recordings will be destroyed once the interview coding has been completed.

INFANT AND CHILD UNDER-5 MORTALITY EVIDENCE BASED INTERVENTIONS (EBIs)