Closing the data-policy feedback loop

If the COVID pandemic has illustrated any principle of public health, it is of the need for data to inform policy. Around the world, health leaders have struggled to contain the spread of a disease on which they too often have little data.

But even before the pandemic, governments and their partners often failed to close the data-policy feedback loop. Exemplars News spoke with experts about why it is so hard to get this right.

Aparna Krishnan is a project director in J-PAL South Asia and leads J-PAL South Asia's Innovations in Data and Experiments for Action (IDEA initiative), which helps NGOs and governments better use administrative data to inform decision making and policies. She outlined three steps towards ensuring data informs policy.

The first is to ensure you are asking the right questions at the right time. “We need to get agreement, buy-in at the very beginning to co-create the questions and a time frame,” said Krishnan. “If you identify what you know and what knowledge gaps you are addressing during what predetermined key policy window.” Second, it is important create interim feedback loops to share insights to speak to current needs and adapt as the context changes. Third, there needs to be a mechanism and capacity for absorbing the data. This includes both NGOs' capacity for curating, packaging, and presenting the data as well as government’s capacity for absorbing this data.

These steps sound deceptively easy, said Krishnan, but are tricky. “This is nobody’s job right now,” said Krishnan. “Everyone is collecting data. There are massive volumes of data becoming available. Health leaders are experiencing an overload of data. But there needs to be a layer between the data generator and the decision maker. The data must be meaningfully organized and made digestible.”

Exemplars in Global Health research highlights a number of success stories in this regard. From Rwanda’s success establishing a data-driven culture to reduce under-five mortality to Ethiopia’s use of data for planning, prioritization, and implementation of new initiatives to reduce under-five mortality.

Perhaps the most illustrative is Bangladesh’s CHW program, which relied on CHWs in the field collecting research as well as surveillance sites such as the International Centre for Diarrheal Disease Research, Bangladesh’s (ICDDR, B), Matlab, just outside Dhaka, which has continually collected data on health trends and shared that data widely. The last piece of the data feedback loop in Bangladesh was the National Institute for Population Research and Training (NIPORT), established by the MOHFW in 1977. NIPORT is an autonomous research institute that conducts public health surveys (e.g., demographic health surveys, maternal mortality survey, health facilities surveys, etc.) and implementation science studies, and disseminates learning from these studies to a broad range of government and NGO stakeholders. The hope is that these leaders then use the learnings to make changes to existing programs or design new programs to address gaps and challenges.

For example, data from Bangladesh’s 2014 Demographic and Health Survey showed that the total fertility rate was much higher in the Sylhet and Chittagong divisions compared to other divisions. NIPORT shared this finding with the directorate general of family planning through a series of national consultations. The ministry then advocated for additional NGO-contracted CHWs to work in these areas and provide information about and access to modern contraceptives.

Another early example of harnessing data to inform policy is Mexico’s conditional cash transfer program, Progresa/Oportunidades. In the late 1990s, the program became a model for CCTs in more than 60 countries, including Peru’s Juntos program. Exemplars in Global Health researchers identified the Juntos program as a critical driver of stunting reduction, in part because of its experimental design and rigorous evaluations.

But even rigorous evaluations do not guarantee impact. Some non-governmental organizations invest precious resources in data collection and research only to see their findings fall on deaf ears. Involving partners in each step of the process, from research design to analysis to dissemination, is critical and can help ensure the results are accepted by all and have the power to inform policy, said Dr. Djoumé Diakité, Mali Country Director for Muso.

“We co-create the research with our government partners,” said Dr. Diakité. “And our technical team works with partners in government to support them in interpreting the data, publishing the data, and presenting the data at government workshops." Added Dr. Diakité, “If everyone is engaged from the first step, it is more likely that everyone will accept the results.”

The World Health Organization, which maintains that countries "should use data and evidence to allocate resources effectively, enhance performance and demonstrate accountability nationally and globally” has created a platform with resources to support the collection of data and its use to inform decision makers.

An increasing number of countries and geographies are committing to systematically using data to improve service delivery and decision making, including the state governments of Haryana and Bihar, in India, as well as the national governments of Ethiopia, and Sierra Leone.