This summer, World Health Organization representatives gathered in Tunis, Tunisia with two lead investigators and a small team of researchers from six African countries to discuss a unique research study. Assembled from August 29th to September 1st, the group – which included staff from WHO headquarters as well as the Inter-country Support Team West Africa – participated in a workshop to provide feedback on on-going data collection and to train researchers on qualitative data analysis using a realist lens.
The objective of the ongoing research is to better comprehend how and under what circumstances the EU-Luxembourg-WHO Universal Health Coverage Partnership (UHC Partnership) contributes to strengthening health policy dialogue towards universal health coverage. The research is currently conducted in Burkina Faso, Togo, Liberia, Cabo Verde, Niger and the Democratic Republic of Congo. The Tunis workshop followed a kick-off meeting convened earlier this year in Ouagadougou, Burkina Faso, to inform and clarify the approach and the method according to the research protocol, and to discuss details of the next steps of the research.
Realist research: what is this all about? Context – Mechanisms – Outcomes
Realist research is an innovative way to look at the complex interventions in the real word, with the objective of finding out how a complex social intervention works (for example, a program in to support health systems strengthening in a given country), in what context (taking country diversity with its various players into account), and how it contributes with its activities to expected and unexpected outcomes. This theory-based evaluation approach tries to unfold complex chains of processes. It aims to shed more light onto why a program faces certain challenges and why it is successful in other areas, by understanding better the contextual factors and by uncovering hidden key elements: for instance, whether values, agendas, and needs are mutually understood by stakeholders; whether trust in a structured and transparent process is ensured; and whether Ministries of Health hold ownership over processes to move national plans forward (mechanisms).
Why is this research project so critically important for the UHC Partnership?
The UHC Partnership, implemented by WHO and financed jointly by the European Union and the Grand Duchy of Luxembourg, promotes policy dialogue to encourage governments to move towards the goal of universal health coverage. The UHC Partnership provides capacity building and technical expertise to Ministries of Health, centered around policy dialogue on national health planning, health financing and effective development cooperation.
In a global health landscape that is increasingly strategic about making decisions (and allocating funds) based on evidence, a nuanced intervention like the UHC Partnership’s poses an evaluation challenge. “It’s complex, it involves numerous actors who negotiate and exchange information amongst themselves. It’s far more intangible, and far more difficult to evaluate, than a clinical intervention or a medication,” says one of the lead investigators, Dr. Emilie Robert from McGill University, Montreal, Canada. It is also incompatible with benchmark indicators like lives saved per dollar. Yet universal health coverage is a major priority of newly appointed Director-General of the World Health Organization Dr. Tedros Adhanom Ghebreyesus. He has stressed at several occasions that achieving UHC is a political choice made by every nation. The UHC Partnership has been eager to strengthen capacities at the national and sub-national level to support countries on their road to UHC; it has also sought to become increasingly effective and efficient, by drawing lessons learned across countries, sharing best-practice examples, and communicating the achievements made so far to further convene key stakeholders (national authorities, policy makers, and donors) to come onboard.
Against this background, the realist approach provides a welcome alternative to typical cause-and-effect evaluation. The focus in realist research is instead on understanding linkages between different elements of a program. “The person who popularized realist evaluation would say that any intervention, any program, does in fact function. It just doesn’t work in all contexts, for all populations, for all beneficiaries,” Dr. Robert explains. “Understanding how it functions, for whom, and how context affects this – that’s the goal. It’s about the how rather than about the if.” To explain the realist approach to her team of researchers, Dr. Robert has a diagram of the UHC Partnership’s work. She points out how few directional arrows link activities to intended outcomes: she is interested in understanding connections in every direction, rather than just identifying an impact.
Investigating these connections requires both breadth and nuance from the study’s team. The national researchers are conducting semi-structured interviews with key stakeholders involved in policy dialogue on health planning, financing and effective development cooperation. They are conducting observational research, by participating in national workshops and negotiations; they are also reviewing all relevant country-level documentation. A review of theoretical literature (on capacity building, policy dialogue, and other topics) complements these empirical data.
In Tunis, over a fast-paced week, national researchers shared their insights about the ongoing data collection. Project leads Dr. Robert (McGill University) and Prof. Valéry Ridde (University of Montreal) further trained national researchers on how to analyze their data using qualitative data analysis software. “It is quite rare to have researchers and lead investigators in the same room, from different countries, discussing the same intervention across countries,” explained Dr. Robert.
In commissioning the study, WHO is enthusiastic about building a realist research skillset in target countries. Most of the team’s researchers, who come from varied backgrounds (e.g. sociology, anthropology, medicine), are using this approach for the first time. The realist approach has been slow to reach the mainstream and is rarely taught at universities. Yet it is evident to Dr. Robert that, as development assistance takes on complex challenges in policy and governance – in the health sector and elsewhere – these new skills will be increasingly solicited.
Data collection for the study is now well underway, but it will be another year before the research is published in its entirety, says Dr. Robert, “Very few realist research studies have been conducted at this scale.” As a result, the findings will not only be valuable to all governance programs within WHO, first and foremost the UHC Partnership itself, but also of interest to realist researchers and evaluators. “The program has been going for five years already,” says Kira Koch, a consultant with WHO on the UHC Partnership. “It’s very much the right time to reflect on how we’re doing and understand where we go from here.”