Methodology

Overview

• The Exemplars in Primary Health Care study aims to systematically and comprehensively identify countries whose primary health care (PHC) systems have performed exceptionally well over the past 20 years relative to their total health spending.
• Exemplar countries were selected through a tiered process combining econometric and descriptive analyses, validated by expert input. To read more about the methodologies that guided country selection and study, please see the Zambia, Ghana, and Peru methodology pages.

Cross-country synthesis methodology

The purpose of the methodology is to detail an approach for synthesizing lessons learned across all five PHC Exemplar countries’ health systems. Between April and September 2023, research teams produced cross-cutting lessons derived from a body of evidence on each of the PHC Exemplar countries. Synthesizing this information from multiple sources into generalized and common insights is intended to increase the applicability of findings and to support transferability and evidence uptake in other LMICs.

The process that research teams took was collaborative, iterative, and nonlinear. Country results were produced via a systems dynamic modeling approach that leveraged group modelbuilding exercises to investigate the interactions and interdependencies between elements, structures, and stakeholders within the PHC system. As a result, the cross-country synthesis was produced in a similar fashion, by leveraging group-based synthesis to generate insights about the most evidence-based and salient messages across the five countries. The below presents the process used to produce final insights.

1. Define a common taxonomy and an approach for its application to classify and compare results.

The Exemplars in Primary Health Care project set out to understand how countries have reformed their PHC systems to address sources of inefficiency and to improve efficiency over time, including details of the subsystems and operations that drove improvement at both national and subnational levels. To present our findings, researchers developed a taxonomy of key terms and system elements for ease of standardization of synthesis across countries.

Researchers categorized and synthesized evidence for each country across each layer of the taxonomy.

2. Develop a list of drivers with supporting evidence.

• In May 2023, researchers responsible for each respective country case study gathered in a workshop to share findings and decide on a core set of drivers common across countries. Researchers leveraged multiple group building exercises, discussions, and expert input to arrive at a final list of drivers (see Figure 17).
• Next, the teams reviewed national evidence and nested individual findings within their respective driver. Given the nature of the interrelation between elements of the system, evidence may be reflected across multiple drivers. Researchers used best judgment to align findings where they would be most salient and relevant for external audiences.
• For the evidence nested across each driver, teams reviewed the taxonomy definitions and began applying their categorization to distinguish evidence classified as an intervention—what was implemented?—or a critical operation—how was this implemented?—to enable further synthesis and insight generation.
• Next, the teams reviewed subnational evidence generated from the district-level protocols implemented in a subset of four districts in each Exemplar country. They produced synthesis tables and findings from within and across subnational units and across countries. Subnational results were aligned with the standardized list of drivers to enable cross-examination of differences and similarities between national and subnational insights and to further supplement driver evidence.
• Finally, the teams graded the strength of evidence within each of the drivers. They leveraged a conservative scoring system to determine the scale by which evidence “satisfied” the conditions of the driver definition.
  • Complete evidence. Evidence of the driver exists across national and subnational results and appeared within all sources of data collection (quantitative and qualitative), including modeling exercises, key informant interviews, and literature review.
  • Partial evidence. Evidence exists only in national or subnational results (not both), it appears in some (but not all) sources of data collection, and there is conflicting information between data sources.
  • No evidence.No evidence exists across national or subnational results, and it does not appear in data collection.

Figure 17: Pathways, drivers, and strength of evidence

3. Distill pathways to summarize how countries achieved health systems change.

The purpose of the pathways is to group interconnected drivers (and their respective reforms and critical operations) to detail how they worked together to improve the efficiency, equity, and quality of their PHC systems. Producing a standardized list of pathways took the following three steps:

• List original systems dynamic modeling pathways by country and compare. Each country case study produced its own set of pathways by which it achieved health system efficiency, equity, and quality. Although similarities exist between countries, no two pathways were the same in driver composition.
• Describe how pathways influence performance. The next step teased out the nuances of how interconnected drivers work together within and across pathways to produce intermediate outcomes within the health system that ultimately resulted in the efficiency gains that made these countries exemplary. For instance, within Pathway 1, “Spending enough, and spending well, on PHC,” Driver 2 or “Financial management” worked together with Driver 4, “Sectoral coordination and multisectoral action,” to produce efficiency gains. Better management of resources, aligned with health sector goals, ensured money efficiently flowed through the system.
• Develop a set of standardized pathways. Last, cross-country pathways language was finalized to accurately capture and reflect commonalities and nuances across countries.

4. Draw out additional insights for transferability.

The purpose of additional analysis was to complement the final storyline of cross-country findings, to ease transferability for key audiences. Results were further synthesized in three methods and were reviewed and validated via discussion and expert input:

• Sequencing of key interventions. To distill insights into how Exemplar countries sequenced PHC system reform over two decades, research teams collated a list of all key interventions implemented during the study period for each country, indicating the approximate year the intervention was initiated and the category to which it mapped within the WHO PHC framework levers. Next, researchers assessed where each Exemplar country was in its PHC performance journey at different times by determining the years in which each country was at low (20–40), medium (40–60), or high (60+) performance according to its UHC effective coverage index measurement. Finally, the aforementioned intervention timelines were mapped against the three phases of the PHC performance journey to analyze sequencing patterns and compare whether countries were implementing similar intervention categories at the same stages of PHC performance. Although research found a degree of variation among Exemplar countries, a three-step sequence was able to be discerned that summarizes the order in which PHC interventions were sequenced.
• Context behind successful reform efforts. Insights that explain additional contextual factors spurring the success of Exemplar countries’ PHC reform efforts were additionally synthesized by reflecting on existing literature and other country experiences to compare the contexts in which reforms were carried out. For instance, research distilled insights about which stakeholders (ranging from donors to multilateral financial institutions to civil society groups) influenced intervention implementation within the study period. This supports other countries’ ability to situate themselves within Exemplar contexts and decide which information is relevant, given the complexities of their own stakeholder environments.
• Policy recommendations. To further support the transferability of lessons from the countries, teams considered which pieces of evidence could be turned into recommendations by adjusting language of key findings and reflections into language that can be acted upon.