Characteristics of study participants
A total of 36 researchers and nineteen decision-makers participated in one or both waves (Table 1). In the first wave, 25 researchers and ten decision-makers participated. In this wave, no decision-makers from Rwanda and Ethiopia were involved as we deemed it unethical to approach them at the height of the COVID-19 pandemic. In 2022, no decision-makers from South Africa were approached for logistical reasons. In the second wave, 27 researchers were surveyed of which seven participated in a focus group discussion.
TABLE 1 STUDY PARTICIPANTS
Table 1 Overview of study participants
|
CEBHA+ researchers
|
Decision-makers
|
|
1st wave
|
2nd wave
|
1st wave
|
2nd wave
|
Total N participants
|
25
|
27
|
10
|
11
|
Survey n responses a
|
24
|
27
|
7
|
11
|
Survey response rate
|
24/26 (92 %)
|
27/32 (84 %)
|
7/14 (50 %)
|
11/16 (69 %)
|
Interview
|
25
|
-
|
8
|
-
|
FGD
|
-
|
8
|
-
|
-
|
Both (survey and interview or FGD)
|
23
|
7
|
5
|
-
|
Country
|
Ethiopia
|
2
|
5
|
0
|
2
|
Malawi
|
5
|
6
|
4
|
3
|
Rwanda
|
6
|
6
|
0
|
4
|
South Africa
|
7
|
7
|
1
|
0
|
Uganda
|
4
|
3
|
5
|
2
|
Gender3
|
Female
|
16
|
13
|
1
|
4
|
Male
|
7
|
13
|
6
|
7
|
Prefer not to say
|
1
|
0
|
0
|
0
|
Highest educational background b
|
Doctorate
|
11
|
10
|
1
|
1
|
Master’s degree
|
10
|
12
|
4
|
9
|
Professional degree
|
3
|
4
|
1
|
1
|
Bachelor’s degree
|
0
|
1
|
1
|
0
|
Professional background b
|
Public Health
|
11
|
13
|
1
|
6
|
Medicine
|
10
|
7
|
3
|
1
|
Epidemiology
|
2
|
0
|
0
|
0
|
Other
|
1
|
5
|
3
|
4
|
Institution b
|
University
|
19
|
18
|
0
|
0
|
Research Institute (not within a university)
|
5
|
7
|
0
|
1
|
Government department
|
0
|
1
|
5
|
8
|
Regional or local health authority
|
0
|
0
|
1
|
0
|
Non-governmental organisation
|
0
|
0
|
0
|
2
|
Time at this institution b
|
One year or less
|
1
|
0
|
0
|
0
|
2 or 3 years
|
8
|
4
|
1
|
0
|
4 or 5 years
|
3
|
7
|
1
|
3
|
6 – 10 years
|
6
|
6
|
1
|
5
|
11 – 15 years
|
1
|
7
|
1
|
2
|
More than 15 years
|
4
|
3
|
3
|
1
|
Overall work experience in the field b
|
5 years or less
|
2
|
5
|
1
|
1
|
6 – 10 years
|
9
|
3
|
2
|
4
|
11 – 15 years
|
7
|
8
|
1
|
4
|
More than 15 years
|
5
|
11
|
3
|
2
|
Legend: FGD: focus group discussion; a participant numbers include incomplete forms (n =1; except in the DM 2020 wave); b based on sociodemographic information in survey responses;
|
Included documents
We included 92 IKT-related internal and external documents (Table 2).
Table 2: Included documents
|
Ethiopia
|
Malawi
|
Rwanda
|
South Africa
|
Uganda
|
Other
|
Total
|
IKT strategies (including revised versions)
|
1
|
3
|
1
|
3
|
2
|
|
10
|
IKT ‘updates’
|
9
|
11
|
9
|
7
|
8
|
|
44
|
Documentation of IKT activities (e.g. meeting minutes)
|
1
|
5
|
0
|
1
|
7
|
|
14
|
IKT team call minutes
|
|
|
|
|
|
17
|
17
|
External documents (e.g. National Strategic Plans for NCDs or Road Safety, funder-initiated evaluation, WHO guideline, transport policy)
|
|
|
1
|
2
|
2
|
2
|
7
|
All documents
|
|
|
|
|
|
|
92
|
Proximal outcomes
We identified proximal outcomes related to capacity-building, relationship-building, and collaborative research.
Capacity-building
Capacity-building is a multi-level concept (47). In our initial programme theory, hypothesised outcomes included only individual-level outcomes, notably ‘access to information and contacts’, ‘broadened perspective and skills’, and ‘capacity for collaboration’. Below, we expand these to capacity-building at the (i) individual and (ii) organisational level as well as capacity-building contributing to an (iii) enabling environment (47).
Individual-level capacity-building
Research and research use skills
Researchers reported mostly positive results with respect to research skills gained (Figure 2). Approximately 80% of researchers indicated that the collaboration had increased their receptiveness to new ideas or evidence ((n=21 (2020), n=25 (2022)); and provided them with an opportunity for personal or professional development (n=19 (2020), n=21 (2022)). Fewer respondents indicated that they had improved their capacity to develop research questions, conduct research, and improve their ability to access relevant research information. In qualitative data, there was little reference to research skills gained.
The majority of decision-makers indicated that the collaboration had helped them increase their research use skills, that is to locate, evaluate, interpret, and apply research evidence (Figure 2). In qualitative data, there was no reference to evidence use skills, except for researchers in Uganda and Rwanda reporting that decision-makers were interested in learning research methods.
FIGURE 2
Figure 2: Research skills and research use skills, in 2020 and 2022, respectively
IKT skills
IKT skills relate to the identification of stakeholders, development and implementation of an IKT strategy and general stakeholder engagement (Table 3). Most researchers already had substantial experience in stakeholder engagement as part of their work routine. They reported high levels of comfort and skills to engage in the research collaboration but few had received prior KT training. At the second wave, 88% of surveyed researchers indicated that the IKT approach helped them to make deliberate decisions about stakeholder engagement, 92% felt comfortable advising colleagues on this, and 77% would recommend an IKT approach in future projects.
IKT skills were not investigated in our survey but featured strongly in qualitative data. In general, researchers from all countries had developed substantial skills in developing and implementing an IKT strategy and honed general stakeholder engagement skills (Table 3). However, skills related to de-prioritisation of stakeholders were only mentioned as important by three interviewees from South Africa. Knowledge of IKT as a theoretical concept and an area of research remained more ambiguous with researchers frequently describing that they were on a ‘learning curve’.
Table 3 IKT skills
IKT skills sub-category
|
Skills
|
Detail and exemplifying quotes
|
Stakeholder identification skills
|
Skills to identify, map, prioritise, and analyse stakeholders
|
‘Yes, so, when we did the stakeholder mapping, we specifically said what we needed each stakeholder for because they have different strengths and […] areas of expertise.’ (MW RS 4)
‘But also, which stakeholders can also negatively influence how your research/ […] how well it's received.’ (MW RS 2)
|
IKT strategy development and implementation skills
|
Skills to choose the appropriate timing for IKT
|
Includes consideration of current stakeholder priorities and timeliness of responses:
‘You can't ignore […] what issues are priorities at that time. So, like a month before the election, people are very, very busy in every Ministry […]/ So, you can't go there and then say, “I want to talk about salt reduction, and, you know.” You have to sense the time.’ (MW RS 4)
‘[…] within CEBHA and also within other projects, the one thing that I’ve learned is around timeliness of responses and engagements and knowing when is the right time to contact and ask for information and also the right time for putting things forward […]. And so I think key to keeping that relationship going is timeliness of information […]/ people at the policy level are inundated with a lot of requests and lots of different researchers coming to them from all different fronts. So you must find ways of not also adding to that burden.’ (SA RS 7)
|
Includes consideration of when not to engage stakeholders:
‘And there’s been moments where we’ve had to make decisions that actually maybe now is not a good time to actually engage that stakeholder about this particular issue because, in a month’s time, we want to engage them about something else.’ (SA RS 4)
|
Skills to choose the appropriate messenger for IKT
|
Included various strategies:
- identifying a team member who was ‘exceptionally good at forming personal relationships’ (Ethiopia)
- assigning messengers based on geographical proximity (Malawi)
- using researchers’ long-standing contacts (all teams)
- enlisting stakeholders as messengers, e.g. lower rank staff within the same organisation to follow-up with senior staff (Uganda)
- accessing new contacts through existing stakeholder contacts (Rwanda, South Africa, Malawi)
‘[… our messenger] varies from our communication to another one. […] For example, [… researcher] who has been working with a specific stakeholder like RBC, the Department of NCDs, [it is therefore] easier to let him continue in that way and keep enforcing the ties that are existing between the two institutions.’ (RW RS 2)
|
|
Skills to choose the appropriate forum and medium for IKT
|
Included skills in developing issue briefs.
Included gauging whether stakeholders preferred further in-person meetings after initial face-to-face meetings and using conferences and trainings strategically as for a for engagement.
|
General stakeholder engagement skills
|
Understanding stakeholder needs and interests
|
Closely linked to skills in stakeholder mapping and IKT strategy development: Understanding of stakeholders’ motivation helped researchers to choose an appropriate engagement strategy. Strategies included:
- highligting of the ‘mutual benefit’ of the collaboration
- showing appreciation for stakeholders’ input, e.g. by involving them as guest speakers
- monetary or in-kind incentives:
‘So for a lot of governmental departments, they will meet you if there's an allowance involved. So if you have a Skype meeting, for example, they don't really benefit anything. We can't get their transport reimbursed.’ (MW RS 2)
|
|
Dealing with staff turnover
|
Ugandan and South African teams strategically established multiple connections to address this challenge:
‘[…] if we are contacting Ministry of Health, […] we should not focus on one person. […] This has been a lesson to us. Never have just one person involved as a stakeholder from one organisation.’ (UG RS 1)
‘And to have multiple members of our staff know about those and be introduced to those relationships, so that we never are in a situation where we have a fractured relationship, […] that's been really important in terms of our strategy, irrespective of who's maintaining the relationships that were not overly reliant on individuals.’ (SA RS 3)
|
|
Dealing with stakeholder power
|
This was actively considered at the stakeholder mapping phase but was otherwise rarely addressed explicitly. Skills in navigating power imbalances in the research collaboration are still evident implicitly through the variety of strategies employed by researchers to deal with this challenge:
- making concessions to high-level decision-makers to accommodate their schedule
- considering hierarchies to choose an appropriate messenger
- balancing of researchers’ support of the decision-maker’s agenda
- managing of expectations regarding such tasks and available resources:
‘[Decision-makers] will be very excited about the project. But then they will have ideas on what it is they want to add. And you're like, "No. […] it would be wonderful to do that. But that's not part of my plan. And I actually don't have the resources for that". So it's something that one has to do. I think be wise. How do you say it without saying only interested in your own activity, which you can fund.“ (UG RS 2)
|
TABLE 3 IKT SKILLS
Organisational-level capacity-building
Organisational capacity-building occurred to a variable extent. IKT staff time was under-budgeted for in four countries, which meant that IKT focal points undertook IKT in addition to other work (Ethiopia, Rwanda, Uganda). In Malawi, an intern coordinated IKT activities. The resulting limited staff time for IKT, in combination with insufficient IKT training and staff turnover, led to persisting uncertainty about the roles of IKT focal points and lack of team buy-in, hampering organisational capacity-building:
‘[…] we got trained [in IKT]. We came back to train others. I mean we were not experts, okay. So […] in the way we transferred the information […] we did not/ were not able to convince people, our colleagues to really take it very serious.’ (MW RS 3)
Efforts to build structures for organisational capacity for EIDM included the development of an evidence-based public health course which was run in all CEBHA+ countries (48). In South Africa, this complemented one institution’s existing KT and issue brief courses (33,34,49,50), which were attended by some CEBHA+ researchers. The Ethiopian team’s collaboration with the new knowledge management unit at AHRI and the KT unit at the Ethiopian Public Health Institute contributed to cross-institutional capacity-building.
Capacity-building for an enabling environment
The Malawi team partially revived an existing KT platform. In South Africa, the NCD symposium (2020) presented an opportunity for a variety of actors to network and access local NCD research. The Ugandan team’s effort to improve road traffic data involved data collection training for police officers. The Ethiopian team initiated a longitudinal training and mentorship for junior researchers who undertook systematic reviews on topics prioritised by the Ministry of Health (MOH). Of note, none of these efforts were CEBHA+ deliverables. Instead, capacity-building for an enabling environment addressed local needs.
Relationship-building
Relationship-building is a process of establishing, strengthening and maintaining relationships. In our initial programme theory, outcomes linked to relationship-building were mutual understanding, trust, and attitudes. These were revised to include (i) appreciation, (ii) trust, (iii) partnership formation, and (iv) access to contacts and networks.
Appreciation
In quantitative and qualitative data, appreciation for the collaboration, the respective partner, and for early engagement featured prominently. Most survey respondents agreed that contributions of partners were valued, that the collaboration incorporated broad perspectives and added value to the research. Agreement was lower for statements about acknowledgement in project reports and of partners’ views (Table 4).
Decision-makers strongly valued being involved by researchers early on in the research process and were content to provide input, which facilitated later work:
‘[Partners were] part of our planning meetings. They were the ones who told us you haven't thought about this, you haven't thought about that, you haven't included this person. So, it was very easy to go back to them once we started collecting data, and say, “[…] we're having a problem in this area. Can you help?” They were very keen to help because they already knew about us.’ (MW RS 4)
Stakeholders appreciated the collaborative approach as it contrasted with usual practice:
‘We hear from [community stakeholders] that the CEBHA project is a good research because it has involved them, it is not the same as other research projects which come and collect data without involving them. They said if other studies are conducted like CEBHA it would be fine. So for them they are appreciating it.’ (RW RS 5)
In Uganda, stakeholders appreciated the CEBHA+ meetings for providing a forum for conversation between various road transport stakeholders who would not normally talk to each.
Table 4: Relationship-building outcomes: appreciation, trust
|
Researchers
|
Decision-makers
|
|
2020
|
2022
|
2020
|
2022
|
|
N (%) Agree
|
N (%) Agree
|
N/total Agree
|
N/total Agree
|
Appreciation
|
|
|
|
|
The individuals involved represent a broad range of perspectives. a
|
21 (91)
|
24 (89)
|
6/7
|
7/8
|
CEBHA+ partners value my contributions. a
|
19 (83)
|
22 (81)
|
6/7
|
7/8
|
All partners are acknowledged in CEBHA+ project documents (e.g. reports, publications). a
|
15 (65)
|
20 (74)
|
4/7
|
6/8
|
The collaboration between stakeholders and researchers in CEBHA+ added value to the ongoing research. a
|
21 (91)
|
26 (96)
|
6/7
|
7/8
|
...I feel that my views are heard. b
|
13 (54)
|
13 (48)
|
7/7
|
6/10
|
Trust
|
|
|
|
|
...the collaborating partners trust each other. b
|
13 (54)
|
13 (48)
|
3/7
|
7/10
|
...I am able to express my views freely. b
|
20 (83)
|
19 (70)
|
6/7
|
6/10
|
Legend: Question format: a yes/no/don’t know; b check all that apply;
|
Trust
Survey participants mostly agreed that they were able to express their views freely in the collaboration but substantially fewer participants thought that collaborating partners trusted each other (Table 4).
Qualitative data provided a more nuanced picture. Without prompting, decision-makers often referred to trust, along with professionalism and respect, as a key relationship characteristic:
‘I think it’s a trusted relationship. It’s a trusted one. It’s both professional and friendship.’ (MW SH 4)
A substantial level of trust is evident from one Ugandan decision-maker’s statement:
‘[I]n most cases I tell [Researcher] what I think, irrespective of what the government position might be. Or the governmental shortcomings might be. So of course I tell her the challenges we are having, and some of them we discuss. Political relationship of this country. […] how road safety doesn’t get an investment.’ (UG SH 2)
Instances of decision-maker initiated engagement, for example, to discuss evidence gaps during the COVID-19 pandemic, reflect that CEBHA+ researchers were seen as trusted sources of information.
In Uganda and South Africa, decision-makers built trust with members of the broader research team over time, beyond the initial senior researcher contact. However, despite individual examples of trust between partners, other relationships were characterised by low levels of trust: survey questions about honest communication about needs, constraints or organisational realities received low endorsement rates. Fear of a ‘hidden agenda’ was frequently referred to by interviewees. Ugandan decision-makers mentioned ‘fear of losing face’ as a reason of why partners may not trust each other enough to communicate transparently.
Partnership formation
CEBHA+ teams formed partnerships with varying levels of partnership strength.
The understanding of ‘partnership’ was ambiguous. Ugandan decision-makers appeared to understand ‘partnership’ mainly as a source of financial support. This may explain why, in 2022, three decision-makers (from Ethiopia and Rwanda) reported that they didn’t think they had established a partnership with CEBHA+ researchers.
Others used the term ‘partnership’ or referred to partners as a ‘team’, being ‘part of the project’, or ‘on board’. Some particularly strong partnerships were rooted in longstanding connections:
‘I'm always available for [Researcher] to call me. I'm always available when [Researcher] calls the workshop. And I don't think I've ever, ever not responded to [their] call.’ (UG SH 2)
Interviewees in South Africa described one decision-maker as their ‘champion’, a term introduced in IKT training:
‘[…] I would say the person who had made CEBHA more […] acceptable and implementable in South Africa, yeah, she was the champion. […] unfortunately, because of COVID, she was also very, very busy, and she really gave more than what a stakeholder of her calibre would, but it's because she had an interest of the kind of work that we're doing, the knowledge, the science.’ (SA RS 2)
In IKT documents partnership formation was evident from the continuous presence of certain decision-makers in project meetings.
Access to contacts and networks
Researchers and decision-makers gained access to a wide range of new contacts and networks. A majority of survey participants agreed that the collaboration had ‘very much’ or ‘somewhat’ improved their access to contacts.
Researchers gained access to EIDM-focused networks, such as Evident and the African Institute for Development Policy in Malawi, and to some high-level decision-makers:
‘And then, we actually […] had access to the Minister of Health who was also at the [South African NCD] symposium and gave a talk […] which is a very (laughing) rare thing to happen.’ (SA RS 1)
Researchers and their institutions were thought to benefit from these networks in the future:
‘Actually, CEBHA could initiate the network, but […] AHRI will be in position to, you know, continue the networking or the relationship with different stakeholders. […] In IKT, you will build a network. It's not only for the CEBHA, but even for my personal career, networking has some impacts.’ (ET RS 1)
Collaborative research
In collaborative research, researchers and stakeholders work together over all or multiple phases of a research project, ideally in a continuous manner. In our initial programme theory, outcomes linked to collaborative research were appreciation, diversity of partners, and continuous involvement, which were reduced to just one sub-category.
Continuous involvement
Researchers interacted with stakeholders most frequently in early and late research phases (28), except in Uganda and Rwanda, where collaboration was more continuous as it included data collection (Supplement 2).
Collaboration commenced during the priority-setting process for the project which involved decision-makers (51) and linked to the existing KT routines of CEBHA+ teams. Some early interactions with decision-makers were mandatory, e.g. requesting research approval. At all sites except Uganda, stakeholder engagement was part of a research method, i.e. for citizen science (52) and for the situational analysis (53,54).
‘[The] situation-based analysis for population-based interventions targeting risk factors of diabetes and hypertension […] required from the onset […] engaging with the stakeholders as we developed the whole data collection tools and […] process.’ (SA RS 2)
Some data collection activities required facilitation by decision-makers, e.g. for accessing health and police records (Uganda) or insurance data (Rwanda). Some collaborative research originated from the strategic establishment of boundary spanners enabling repeated interaction: decision-makers had been asked to join the project advisory board (Uganda, South Africa) or undertook research on CEBHA+ PhD studentships (Malawi, Rwanda).
Intermediate outcomes
Intermediate outcomes emerge from the interplay of the proximal outcomes. In our initial programme theory, we had only conceptualised decision-makers’ attitudes as intermediate outcomes. We revised this as changes in knowledge and attitudes among both decision-makers and researchers.
Changes in knowledge
This included decision-makers learning about (CEBHA+) research and researchers learning about the decision-making context. In the survey, a majority of researchers and decision-makers indicated that the collaboration had increased their knowledge, understanding or views about the health issues addressed in CEBHA+ and enhanced their confidence in day-to-day activities (Figure 3).
FIGURE 3
Figure 3: Researchers’ and decision-makers’ changes in knowledge, in 2020 and 2022, respectively
Fewer survey participants thought that the collaboration had improved their access to relevant information.
Contrastingly, interview data indicates that researchers actively sought and accessed information. This strongly improved their understanding of the decision-making context, e.g. regarding political mandates, priorities, and post-election restructuring, and subsequently informed research activities:
‘What I understand from the ministry is […] they have deliverables that they have to put through, but they don't have the resources to do so. So [when] we kind of give them the platform […] or rather ask them what they need from us, it kind of opens up […] It makes it easier for us to get our work done when we also put forward their agenda.’ (MW RS 2)
Seeking to understand decision-making priorities, the Ethiopian team conducted a needs assessment among MOH staff.
Decision-makers’ knowledge of relevant research features less prominently in interviews. In one case, they requested results from a systematic review but did not learn much as the review was ‘empty’. Still, seven out of eight decision-makers felt that they had become more knowledgeable about the scope of work at the research institution (2022 survey). The IKT approach further enhanced the visibility of the research institutions.
Across all groups and sites, gaining a new perspective was emphasised as a key outcome:
‘[…] my interaction with the researchers […] it really opened up my mind. It pushed me to think more, to perceive things differently.’ (UG SH 3)
Changes in attitudes
We identified subtle changes in attitudes towards research evidence (decision-makers) and more pronounced attitudinal changes regarding research impact (researchers) and research collaboration (both groups). These were typically linked to existing, predominantly positive attitudes.
Decision-maker attitudes
Decision-makers who were aware of CEBHA+ evidence generally valued this evidence and indicated that they were intending to use it (Table 5).
Table 5: Decision-makers’ attitudes towards research evidence
|
|
2020
|
2022
|
|
The research evidence produced in CEBHA+...
|
Agree N / total
|
Agree N / total
|
Value
|
... is relevant to the health issues addressed by the partnership
|
7/7
|
6/6
|
|
... is directly applicable
|
5/7
|
5/6
|
|
... is trustworthy
|
7/7
|
6/6
|
|
...more likely to be used in decision-making
|
6/7
|
6/6
|
|
...more likely to have a lasting impact on public health
|
6/7
|
6/6
|
Intention
|
I intend to use CEBHA+ research evidence in my work.
|
6/7
|
9/10
|
Consideration
|
Being engaged with CEBHA+ increased my active consideration of research evidence in my work.
|
7/7
|
7/10
|
In qualitative data, decision-makers’ interest in research was apparent but this largely seemed to pre-date the CEBHA+ project.
‘[…] through some of the stakeholder's meetings, we [heard that], they're eager to get the […] study results, […] and then to see what they can improve because in the Rwandan context, […] from the administration and governance level, they are looking for anything that can [improve] what they are doing.’ (RW RS 1)
It was also evident that attitudes towards research collaboration had improved among some stakeholders, often due to persistent efforts by researchers:
‘[Eventually, the road traffic insurance companies] were very happy to help us, […] even someone […] who didn’t want to talk to you, they are very excited to give us information and to help us […] because it’s in their big interest […] And that’s like the approach we used to get them on board. They were like, “So at the end of the day, […] we are the ones who are going to gain from this project.”’ (RW RS 6)
This was echoed in survey data: All decision-makers surveyed in 2022 indicated that they would reach out to the CEBHA+ researchers for future queries.
Researcher attitudes
Many researchers embraced the ‘IKT idea’ as a vision for research impact and became more interested in collaborating with decision-makers. Research impact was seen in both visionary and practical terms:
‘It makes you […] remember not to do things in a bubble. Always consider what your long-term objectives are. So don't do research for the sake of doing research, but always to think long term and to think who else needs to know about this. What input can they give?’ (MW RS 2)
Among some researchers, this attitude was coupled with great openness to taking on decision-maker insights, e.g. when co-developing a logic model for integrated models of care to ensure it was relevant to both the South African and global context. Two senior researchers were described as only becoming interested in (I)KT and research collaboration through CEBHA+ and one Ethiopian researcher reported change at the level of the institution, where the benefits of systematic stakeholder engagement became recognised:
‘So, now the other projects or other directorates under AHRI, they are consulting us. Or they are using us as a, you know, a link to the different stakeholders.’ (ET RS 1)
The interest in IKT is also reflected in researchers adopting IKT in other projects, such as proposals and teaching (Germany, South Africa).
Unintended and unanticipated effects
The opportunity cost of doing IKT was an unintended – yet foreseeable – outcome discussed by some researchers, leading to potentially overburdening decision-makers in resource-constrained settings:
‘[I]n some ways, CEBHA has been trying to reinvent some wheels […] there’s already a lot going on in [our country] that could’ve been built on […] because the policymakers that people are trying to engage with are all busy people. […] if the people keep saying, “[…] we’re going to do policy engagement. Come to our stakeholder meeting now […].” I think that has real opportunity costs in terms of trying to take people’s time up that way.’ (RS[2])
Within the broader EIDM system, the IKT approach sparked the research funder’s interest in research impact, which was not anticipated: The CEBHA+ IKT approach was mentioned multiple time as a good practice example in the funder-initiated midline evaluation and ‘policy engagement’ became mandatory in subsequent funding lines.
Distal outcomes
Increased use of contextualised research evidence in decision-making was initially conceptualised as the final outcome linked to the IKT approach. We rephrased this category, added the researcher dimension, the production of relevant research evidence, and adapted the decision-maker dimension to consideration of contextualised research evidence, as evidence use proved conceptually challenging to evaluate.
Production of relevant research evidence
Due to the small number of study participants, our insights into decision-makers’ views on CEBHA+ outputs remained limited. In the survey, the majority of decision-makers agreed that CEBHA+ research evidence was presented to them in an accessible format, an understandable language, and a timely manner.
Two CEBHA+ projects were highlighted by decision-makers as important solutions to local priorities, for example:
‘So the great initiative for diabetes that was done by [Researcher], that was important because we are not counselling our diabetic clients as we are counselling our HIV positive clients. [..] But both of them have significant morbidity and mortality. So he came with a group education initiative for diabetic patients. So that's something that we needed.’ (SA SH 5)
This study subsequently informed the local department of health and guidelines in Ethiopia, Malawi, and Rwanda. Ugandan researchers and their partners identified the lack of digitised, valid road traffic crash data as a major gap. This led to a joint pilot project digitising crash data from police and healthcare archives, which had not been planned in the original proposal. One decision-maker explained that CEBHA+ research was highly applicable but recommendations were not implemented:
‘I have seen the benefits [and] the value that they could bring to the sector, to the prevention of road crashes. But yeah, I haven't had the opportunity to implement them. […] so the applicability of research findings is they can really be done.’ (UG SH 2)
Additionally, beyond what was planned in the original proposal, CEBHA+ funding enabled conducting ten rapid reviews on prioritised COVID-19 topics in South Africa and five systematic reviews on various MOH-prioritised questions in Ethiopia.
Other qualitative data pointed to more ambiguity regarding the relevance of research evidence produced in CEBHA+. In Rwanda, the CEBHA+ team used old NCD data due to the tight project timeline despite the MOH requesting use of newer data. Similarly, a South African researcher was only able to interview very few decision-makers for their situational analysis, limiting its relevance. In Malawi and Ethiopia, the CEBHA+ priorities defined with some decision-maker involvement in 2013 (51) did no longer match current research priorities:
‘We had already met with the Ministry of Health and told them about our community level screening […]. And they didn't say a thing. And then when we found out […] that they had been doing this already for three years in Addis Ababa. So I mean our project […] was, well, yes, it was sort of aligned with the Ministry of Health [initially]. But it was already behind [when it started].’ (ET RS 2)
Few scientific articles were published until the end of the funding period. Despite this, CEBHA+ researchers were able to produce a range of relevant outputs (31).
Consideration of research evidence in decision-making
Decision-makers self-reported conceptual, instrumental, and tactical evidence use (55) in the survey. Research evidence was considered by decision-makers when they consulted CEBHA+ researchers: Ugandan CEBHA+ researchers were involved in the development of the Road Safety Action Plan (2021/22 – 2025/26) and were invited to present research in a Parliamentary meeting on road safety, prior to the amendment of the Traffic and Road Safety Bill (30th March 2023). South African CEBHA+ researchers and their ‘champion’ were involved in developing the National Strategic Plan for the Prevention and Control of NCDs (2022 - 2027).
‘A key ally throughout the CEBHA+ project, [decision-maker] applauded the work of CEBHA+ and linked its relevance to the National Strategic Plan for NCDs. She stressed the importance of research collaboration and the need for a coordinated platform.’ (SA policy dialogue minutes)
One study by a Rwandan CEBHA+ researcher was cited in the National Strategy and Costed Action Plan for the Prevention and Control of NCDs in Rwanda (2020 – 2025) (56).
Revised programme theory
The adapted programme theory includes revised sub-categories. Intermediate and distal outcomes were supplemented by the researcher dimension (Figure 4).