How experts use data to identify emerging COVID-19 success stories
EGH’s goal is to help inform current and future policies and programs in global health based on what has worked in the past. We do this by rigorously analyzing the success of countries that have made extraordinary progress. In the midst of a constantly evolving pandemic like COVID-19, it is not a simple matter to identify countries that have been most effective and therefore have the most to teach the rest of the world about best practices. Based on the current data, we developed a methodology to help us identify these emerging success countries. There are considerable limitations to the selection methodology due to the evolving nature of the pandemic including incomplete data, evolving case definitions, and the fact that the ultimate outcomes are unknown.
As you will see, it is not possible to identify emerging success stories, or exemplars, using just one indicator. Each data point has nuanced drivers and meanings, making it important to triangulate and look across multiple indicators to identify countries that have had success to date in managing the pandemic. The daily rates of confirmed deaths follow very different trajectories in countries. The steeper the slope of the curve, the faster the rate of increase in deaths. Countries at the top of the first graph below have had the most deaths, but some of those countries also have larger populations. The second graph below shows deaths per capita to account for differences in population. These graphs suggest that some responses have been much more effective at reducing deaths from COVID-19 than others, but they do not tell us why they are more effective.
Detect, Contain, Treat
To help shed light on why some responses are more effective, we initially started with a four-part framework for epidemic preparedness and response: prevention, detection, containment, and treatment. Because transmission of COVID-19 is ongoing, it is too soon to determine if any country will ultimately succeed at prevention, so we excluded the prevention phase from our analysis. We selected multiple indicators for the detection, containment, and treatment phases that could help us identify which countries are excelling at any given phase. This enables us to identify countries that show emerging success at each phase, which makes it more likely to glean detailed insights that will be useful for other countries.
On this page, we walk through a total of nine graphs addressing all three phases, describing what each graph can and cannot tell us about a country’s response. These graphs are updated daily, so the countries that stand out now might change over time. Some countries with early positive outcomes were unable to sustain their success, whereas the situations in other countries gradually improved. To see the results on any given day of the pandemic or to reflect any range of dates, adjust the slider bar below the graphs. This will turn single data points into line graphs.
Country Selection Criteria
To ensure that our process is rigorous and that other countries can use lessons from our exemplars, we applied the following three criteria to select countries for our analysis:
- We included only countries with populations greater than 5 million. This minimizes the likelihood that apparent trends are background noise in the data, and to ensure we can extract lessons applicable to the many large countries that are currently struggling to contain the pandemic.
- We included only countries in which at least 21 days had passed since the 100th confirmed case. We wanted to include only countries whose response had been going on long enough and with sufficient numbers of cases to be properly assessed, recognizing that in some countries, low prevalence may be the result of effective containment.
- We included only countries with high-quality data on testing, cases, and deaths. Many countries have been slow to publish testing data, preventing proper analysis at this time. Our testing database has been developed with a clearly documented checklist of criteria for inclusion; definitions and units; and detailed descriptions for each country so users can understand the comparability of data from different countries.
The number of countries meeting the second and third criteria will grow over time, increasing the number of countries that can be included in the analysis.
We selected three indicators for the detection phase, all of which are measured relative to population size: tests per capita, tests per confirmed case, tests per confirmed deaths.
In the early stages of an outbreak, containment may be possible by testing frequently to identify cases and trace contacts. However, once community transmission is widespread and containment is no longer possible, population-wide measures to reduce transmission are less dependent upon testing. If these measures are successful in reducing the spread, then case finding and contact tracing eventually become feasible again, and widespread testing again becomes critical.
The bar graph below shows the average number of tests performed per confirmed case. Countries with the highest ratio of tests to confirmed cases are likely to have a more accurate understanding of how many total (not just confirmed) cases they have.
Some testing strategies focus on high-risk groups such as health care workers or high-risk locations such as nursing homes. Strategies that focus on those at the highest risk will result in a lower number of tests performed per confirmed case, meaning countries will know less about of the true magnitude of the outbreak in the community at large.
The graph below shows the total number of tests performed and the total number of confirmed deaths, per million people in a country. In general, a high number of tests per death is preferable because it indicates widespread testing and assessment of community transmission. Considering the context, however, is especially important to ensure we are aware of what the data can and cannot tell us. In very large countries, for example, the evolution of the epidemic may vary across different cities and regions. Some outbreaks may be successfully contained, whereas others are not. These data do not show testing capacity at the sub-national level.
We selected four indicators for the containment phase, relative to the population size: cases per capita, cases doubling time, deaths per capita, deaths doubling time.
The graph below shows the total number of confirmed COVID-19 cases per million people. The number of cases, however, are dependent on a country’s testing capacity and strategy. Low numbers of cases could reveal successful containment or they could indicate an inability or choice not to test widely. A steep slope could indicate high levels of transmission, a rapid increase in testing capacity, or a change in testing strategy to include, for example, asymptomatic or mildly ill people.
The graph below shows the total number of COVID-19 cases per million people and the doubling time of cases. A longer doubling time indicates a lower rate of transmission. The doubling time is less dependent on testing capacity as long as the capacity does not change over time—which it often does. We would consider a country with few confirmed cases and a long doubling time to be successfully containing the outbreak, although low testing capacity could still be a factor. In many cases a long doubling time may suggest that containment measures are starting to take effect. On the other hand, few cases and a short doubling time may suggest that the outbreak is spreading quickly, and the number of cases is set to increase.
We selected one indicator for the treatment phase: case fatality rate (CFR). This phase is extraordinarily challenging to assess because the quality of the data depends on the accuracy of the case data, which in turn depends on testing capacity and strategy. In some countries where testing is effective, the number of confirmed cases may be close to the true number of infections. In other countries, the true number of infections is probably much higher than the number of confirmed cases. Eventually, we will be able to compare the less biased indicator of excess mortality, which measures how many more people died in a given country over a given period than was expected. We will also have better data about the true number of infections, which will allow us to use infection fatality rate (IFR). However, those data are not currently available for most countries.
The graph below shows CFR, which is the ratio of deaths to confirmed cases, or the proportion of people diagnosed who die from the disease. Our assumption is that countries with lower CFRs have more effective responses during the treatment phase. In countries where health systems were unable to meet increased demand for care, CFRs might increase, although we are unable to determine whether that has happened from this data set.
Analysis of CFR data comes with certain caveats. For example, if only the sickest people in a country are tested and therefore diagnosed, CFRs may be artificially high. CFRs may also be strongly influenced by the demographics of the population, with higher CFRs seen in countries where the population demographics are skewed toward groups who have a higher risk of death due to COVID-19, including older people or people with other health problems, such as diabetes and hypertension.
The graph below shows CFR broken out to show the total number of confirmed deaths from COVID-19 versus the total number of cases. This comparison helps provide more context about the drivers of CFR. If the CFR is high, is that because deaths are high or because cases are low? Conversely, if the CFR is low, is that because deaths are low or cases are high?
In early May 2020, we surveyed the countries that appeared to be controlling the outbreak by each indicator and phase as discussed above. Once we had the quantitative results, we also considered diversity and representativeness (e.g., demographics, form of government, geographic features, or global region).
In addition to these three countries, we will continue to refine our methodology and identify more exemplar countries as the pandemic evolves and as more and higher-quality data are available. For example, many African countries did not meet our inclusion criteria because transmission is still less widespread in those countries, but unfortunately that may change soon.
The charts and graphs are updated daily. Our approach and graphs are intended as tools for you to run your own data analysis to seek answers to your specific questions.
We recognize several limitations to this country selection process. The dynamics of the COVID-19 pandemic are different in each country, are evolving at different rates, and at times are dependent on factors beyond the country’s control. These factors include the population density and patterns of geographic dispersion, the number of borders shared with other countries, the number of air and land travel entry points, and the volume of travelers and trade. For example, the challenges of controlling an outbreak in an island country like New Zealand are much different than in Switzerland, a country in the geographic heart of Europe that shares land borders with five countries, with daily border crossings from all sides.
About the authors
|(i) UK Public Health Rapid Support Team||The UK Public Health Rapid Support Team is funded by UK Aid from the Department of Health and Social Care and is jointly run by Public Health England and the London School of Hygiene & Tropical Medicine. The University of Oxford is an academic partner. The views expressed in this publication are those of the authors and not necessarily those of the UK National Health Service, the National Institute for Health Research, or the Department of Health and Social Care.|
|(ii) Our World in Data||Our World in Data is an open science publication which focuses on the world’s largest problems, and publishes the data and research to help make progress against them. The aim of the project is to make this knowledge accessible through the use of interactive data visualizations and clear summaries of the research. Our World in Data is a collaborative effort between the researchers of the Oxford Martin Programme on Global Development at the University of Oxford, who are the scientific editors of the website content; and the non-profit organization Global Change Data Lab, who publishes and maintains the website.|