TCI uses the Most Significant Change (MSC) technique as a qualitative monitoring method in order to gain a fuller understanding of not just what the project is accomplishing but also how. For this analysis, four members of the TCI research team conducted secondary analysis of all of the MSC stories collected in the first year of implementation of the method.
Data collection
The research team developed a semi-structured interview and focus group discussion guide informed by the MSC technique [6]. Data collectors who also served as TCI coaches interviewed stakeholders through purposive sampling. The stakeholders interviewed represented those adapting and implementing evidence-based family planning interventions and those responsible for the oversight and financing of family planning programmes at the city, municipality, county, or state level, depending on the location’s particularities. Some family planning clients and TCI project staff were also interviewed.
For this analysis, we included MSC stories collected between July 2018 and September 2019. The specific timeframe for data collection differed by region, varying from 6 months in Francophone West Africa to 15 months in India, based on when training on the MSC method occurred. In general, this timing corresponds to the second year of implementation of TCI-supported evidence-based interventions. In total, 118 MSC stories were collected during this time period. Of these, 96 MSC stories met inclusion criteria: they included information on the situation prior to TCI’s involvement, the change as a result of TCI’s involvement, and the significance of the change.
Data management and analysis
The interview teams either audio-recorded and transcribed or took detailed notes of the interviews and focus group discussions using an interview guide. Interview transcripts were translated into English when another language was originally used (e.g., Hindi or French).
For this analysis, the research team first transferred all of the stories into a Google Sheet to track and organise the transcripts by country/region and other related characteristics, such as job title and sex of interviewees. Then, each of the four researchers coded six transcripts manually to become familiar with the content and develop an initial set of codes. The team coded phrases and sections within each story to summarise the data, electing to use process codes; process codes employ gerunds (-ing words) to draw out observable and conceptual action in the data [7]. This form of coding seemed particularly relevant in this context because the intention was to extract the actions and consequences of programme planning, implementation, and scale up. The team generated the codes inductively—that is, the codes were created and iterated based on the data rather than predefining the codes a priori—and maintained a codebook of all the codes, along with the definitions. From the small sample of test stories, the research team identified initial codes that were iterated throughout the analysis of the 96 stories. Two of the researchers then coded all of the stories in Atlas.ti software (version 8.4.4) to organise the data under codes. To achieve intercoder reliability, the two members of the research team working in Atlas.ti each coded 20% of the stories separately and reviewed each other’s work to come to consensus before dividing the rest of the stories between themselves.
Ultimately the analysis included 55 unique codes. The research team identified the most frequently used codes and grouped them into emerging themes.
Findings
Of the 96 stories that were included in the analysis, nearly half (49%) were from the Nigeria hub, while East Africa and India each contributed about one-quarter of the stories, and Francophone West Africa contributed four stories (Table 1). The disproportionate collection of stories was attributable to the timing of training and delays in rolling out the MSC method due to staff turnover in Francophone West Africa. Interviewees represented a range of roles in the health system, from government stakeholders (27%) and service providers (20%) to TCI managers (12%) and community leaders (9%). Twelve percent of the stories were from the client’s perspective. The interviewees were relatively evenly split between women and men, with women making up 53% of interviewees. Some stories were compiled through group interviews and thus included more than one interviewee.
TABLE 1. Distribution of MSC Stories by Characteristic (N=96)
|
Regional Hub (where stories originated)
|
No. (%)
|
|
Nigeria
|
47 (49)
|
|
East Africa
|
24 (25)
|
|
India
|
21 (22)
|
|
Francophone West Africa
|
4 (4)
|
|
Role of interviewee in the health system
|
|
|
Government official
|
26 (27)
|
|
Service provider
|
19 (20)
|
|
Health client
|
12 (12)
|
|
TCI state or city manager
|
12 (12)
|
|
Community leader
|
9 (9)
|
|
Community health workers (CHWs)
|
7 (7)
|
|
Health educator/social mobilizera
|
5 (5)
|
|
Implementing partner
|
3 (3)
|
|
HMIS/M&E officer
|
3 (3)
|
|
Other facility staff
|
2 (2)
|
|
Sex of interviewee
|
|
Female
|
51 (53)
|
|
Male
|
38 (40)
|
|
Multiple storytellers
|
7 (7)
|
Abbreviations: HMIS, health management information system; M&E, monitoring and evaluation
a Note: Health educators are government employees, but they are grouped with non-governmental social mobilizers since both health educators and social mobilizers provide information/counselling to clients, but not services.
From this analysis, five key themes emerged as aspects of TCI’s business unusual model that facilitate implementation and lead to positive changes in scale up and sustainability of family planning programmes, according to TCI’s stakeholders: (1) strengthening local capacity, (2) shifting mindsets toward local ownership, (3) improving government health systems, (4) improving data demand and use, and (5) enhancing coordination of partners. In general, the codes applied to the MSC stories in this analysis appeared across the regional hubs, but some codes featured more prominently in certain regions. We explore each of these themes and highlight notable regional variations in the sections below.
- Strengthening local capacity
The most frequently mentioned themes in the MSC stories related to capacity strengthening, with about one-third of all stories highlighting various elements linked to capacity strengthening. This trend was more prominent in India, where many of its stories described changes in local capacity. Specifically, those stories pointed to the value of embedding and aligning TCI tools and evidence-based interventions within existing city structures and processes to facilitate their adoption and to help government staff, especially at the facility level, perform their jobs better. For example, a service provider in Uttar Pradesh, India, explained:
This [data reporting tool] made it possible to review the health center’s family planning data during monthly government meetings … and take corrective actions including arranging human resources and family planning supplies. Having accurate data in a simple format that was being reviewed regularly encouraged better performance and motivated the facility staff and community health workers to give their best.
Many stories described various coaching, mentoring, and training opportunities that stakeholders received from TCI, which focused primarily on city-level implementation of evidence-based family planning interventions. Thematic analysis elucidated the characteristics of TCI’s coaching approach that they value most. For example, a government official in Bauchi state, Nigeria, referenced consistent access to coaching support:
TCI … is different from other projects [in] that the [TCI] office is embedded inside the agency. In fact, they are like staff of the agency, and so collaborating with the staff of the agency in order to provide technical support needed is within the same environment.
Furthermore, TCI stakeholders noted the outcomes of coaching support related to broader health systems functions, such as leadership, management, and coordination. A government official from Dar-es-Salaam, Tanzania, explicitly linked skill building in management, budgeting, and monitoring and evaluation to improved government leadership and service to the community:
We, the government staff, are now involved in planning, budgeting, monitoring, documenting, and evaluation... Right now, planning is done at both facility and municipal levels. Through TCI, there are so many improvements and our activity and budget plans are smart. Most of [the] challenges we used to face have been minimized. The local government knows the community needs and we now have a sense of ownership and we aim at sustaining it.
- Shifting mindsets toward local ownership
Another frequently occurring theme across the region was changing mindsets about the important role of local governments and communities in leading family planning programmes. Altogether, about one-third of the stories covered mindset changes, and the stories from East Africa and Nigeria illustrated such outcomes more prominently than the two other regions. For local governments, the mindset shift was linked to increased political and financial commitments. At the community level, the analysis revealed how community stakeholders recognized the essential role they could play in improving the health of their communities. In both cases, the mindset changes were noted as a foundational step to motivating action among local government players and community members to improve access to and use of family planning services.
Local government players
Local government stakeholders often described TCI’s approach to locally driven solutions as a novel one that they were unaccustomed to—one that they embraced and that prompted them to take action. For example, a health educator in Rivers state, Nigeria, described how empowering this mindset change was:
We are seeing an NGO coming to say that “you drive the process and we follow.” We are seeing ourselves in the driver’s seat and if we don’t do it, no one else will! It calls for ownership and participation at the state level, LGA [local government area] level, and community level. That is a very great change.
In Uganda, a service provider celebrated the mindset shift and movement toward increased ownership and accountability among city government, commenting:
Nowadays, our political leaders are interested in knowing how much of the geography money we are spending in family planning.… Before TCI came on board, these leaders were not asking such accountability-related questions.
Several stories described how stakeholders witnessed the impact of TCI’s demonstration of its evidence-based approach, “dedicated day for family planning services,” in which trained staff, equipment, supplies, and commodities were made available on a pre-announced day and time at an urban primary health center. A service provider of Uttar Pradesh, India, declared:
Family planning was the last thing on anyone’s mind at our urban primary health center. However, after observing and participating in the special drive [for family planning services] in 2018 facilitated by TCI, I saw people coming in for family planning services. From that day onwards, we are regularly conducting dedicated service day without the support of the TCI team.
These mindset changes also spur local stakeholders’ involvement in better understanding and responding to community needs. For example, in Plateau state, Nigeria, a government official shared how one community issue was resolved:
When the deputy governor’s wife and governor’s wife were commissioning the facilities, we learned that they [the facilities] didn’t have water.… We dug a borehole and put a tank in place to pipe the water. I was able to do this with money that the state already had available. We didn’t know that water was an issue before TCI. The TCI model gives us an opportunity to enter to see what some of the problems are.
Community members
Many stories, in particular those from Nigeria, described the community as a key contributor to and implementer of TCI’s evidence-based interventions. This support or engagement with community structures has not only improved the family planning knowledge of community members but also changed their negative attitudes and perceptions of services and empowered them to take action to further help improve service delivery in their communities. For example, an implementing partner of an NGO visited Bauchi state, Nigeria, to learn from its experience implementing the clinic makeover approach and then used that learning to replicate the approach across Gombe state, a non-TCI supported state in Nigeria. A key lesson learnt that the partner took away from the experience was the profound change that he saw within the community:
The work continued to the next day, which is Sunday … with a renewed commitment from the community representatives and the staff including the artisans themselves. In fact, you can see clearly from their faces because they have that impression of owning the process, it is their facility, it is their families who are the direct beneficiaries.
Some stories described how TCI’s evidence-based interventions targeted key influencers, such as traditional and religious leaders and community members who served as community health workers (CHWs), to become family planning champions. Following an engagement with TCI, a religious leader in Plateau state, Nigeria, explained that he was carrying out a door-to-door campaign among his network to share the importance of family planning and that he was sharing what he learned from TCI with other religious leaders, which sparked a broader mindset change among the Muslim clerics in his area:
There were misconceptions on the part of the Islamic clerics, they normally interpret child birth spacing as controlling population but after our interaction with them using the Islamic Perspective and Sermon notes on child birth spacing [from TCI], a lot of them have now understood that spacing in between births for the health and well-being of the mother and child is also promoted in the Holy Quran.
In several stories, CHWs from India shared their concerns about counselling on family planning during household visits. After receiving coaching support from TCI, the CHWs reported that their images improved within their communities as women began to seek them out for family planning information and services. A CHW in Odisha, explained:
Since I had never done this before, I thought that the situation is going to be very embarrassing in the field and I may lose my good connection. … These [coaching sessions] were extremely useful. Now I hold these conversations independently, and rather enjoy doing so because I find that women are making their own decision about contraception.
These mindset changes in the community extended to youth as well. For example, in Benin, a government official highlighted the role of youth in implementation of AYSRH programmes:
The youth in general and young leaders are more and more responsible; as you see them taking part in activities, you can see that they are willing to do their best.
- Improving government health systems
Improving government health systems was another frequently captured theme. Stakeholders illustrated various examples of how TCI’s evidence-based interventions had been institutionalized, diffused, or scaled across the health system. About half of the stories in India illustrated specific examples related to this theme. For instance, the local government, with coaching support from the TCI team in India, mapped and defined slum catchment areas to better define service needs and allocate resources. All government programmes, including family planning, immunization, and maternal and newborn health, started using the slum maps to plan urban health interventions and reported that the mapping and listing intervention had resulted in the strengthening of the urban health system. Additionally, a number of MSC stories from India highlighted how TCI has helped to streamline government systems to process timely release of incentive payment to CHWs, which has resulted in increased motivation among the workers. A CHW from Madhya Pradesh explained the significance of this change:
Earlier, we were not aware of our incentives or responsibilities.… Once [TCI] intervened, our meetings were formalized with our health center staff every Saturday. Now, we are regularly being updated about our programs and incentives.… We are well-versed about the health worker diary [register] and the processes of voucher filling and submission, which we did not know earlier.
Several MSC stories from Nigeria described how the improved facility infrastructure from clinic makeovers supported delivery of not only family planning services but also primary health care services overall. A stakeholder from Plateau state, Nigeria, explained that the government recognized this and supported replication of the clinic makeover approach in other non-TCI supported health centers.
Analysis of MSC stories also revealed a more effective use of existing public health sector staff. For example, a service provider in Ziguinchor, Senegal, highlighted the impact of universal referral, an evidence-based intervention that TCI coached the government on how to implement, which involves maternal and newborn care staff screening their clients for family planning services:
Through TCI tools, a client may now be identified as needing family planning services. She is systematically given the referral sheet for necessary care and when she comes back, she is identified as having received FP services. This strategy did not exist in the past.
- Improving data demand and use
Many stories, especially the ones from India and East Africa, presented a case about improving data for decision-making. These fell into three categories: (1) how TCI used data as part of its advocacy efforts to motivate and inspire local governments to prioritize family planning interventions; (2) how TCI strengthened the capacity of local stakeholders to collect, analyse, and use data; and (3) how this capacity strengthening in data use ultimately led to a culture of data demand by both local government stakeholders and implementers at the facility level.
Data-informed advocacy
MSC stories described a number of approaches to foster local ownership and investment and how those approaches were critical for implementation and scale. One approach involved direct advocacy to local governments to add a budget line item for family planning and AYSRH—and to release those funds. These advocacy efforts resulted in local government allocation and release of funds for family planning programmes in general and for AYSRH programmes specifically.
In addition, several stories described how data was used to demonstrate success of the evidence-based approaches, which then spurred systems-level changes. For example, in Uttar Pradesh, India, several stories described how TCI demonstrated the success of a dedicated day for family planning services in 25 demonstration sites over a two-month period. Upon reviewing the data demonstrating primary health centers’ capacity to provide quality family planning services, the local government deemed the dedicated family planning day a worthy investment and scaled the approach throughout all urban primary health centers. One government official noted:
When we find out that something [is] good and workable [and] produces results quickly, we take it into the system—which is what we have done in case of the dedicated day for family planning services. This is now part of [the] center’s charter and [is] going to sustain forever. The system works, not individuals. So, when something is introduced or added into the system, no one needs to worry about its sustainability.
Data-improved capacity
Many of the stories referred to the significance of the coaching support that TCI provided to its public sector counterparts in a number of areas, a key area being capacity to collect, interpret, and use data. For example, a monitoring and evaluation officer of Bauchi state, Nigeria, shared:
My capacity has now been built not only on how to log into the HMIS platform but how to download data, analyze, and compare it to see the differences and improvements. Not only that, my capacity has been strengthened that I collate monthly data from health facilities and conduct data quality assessment and share it with implementing partners. It wasn't like this before the coming of TCI. I feel very confident now.
Similarly, a health educator in Uganda reported improved capacity among all community outreach workers in her district:
The Iganga District Health Team noticed a huge improvement in the capacity [of the community outreach workers] to compile and submit their reports, including those of family planning. Many of the health facilities have now incorporated VHT [volunteer health team] data into the HMIS monthly health facility report and reporting is on time. VHTs are now in position to cross check in the HMIS to ensure their data is incorporated in the health facility report.
A culture of data for decision-making
Several stories highlighted the use of both TCI-generated reporting tools and a government-owned data platform as ways to promote a culture of data use for decision-making. These efforts improved data accuracy and timeliness of health information data, leading to greater government ownership of the evidence-based interventions supported by TCI. For example, a health information focal person in Uganda shared how the integration of data from CHWs into facility registers, which was reported into the data platform, has helped target services more effectively:
Nowadays, health workers are more responsible and health facilities own their data. They ensure completeness of their FP [family planning] data reports so that their efforts are realized. Community health workers have learnt to demand for their data to be displayed at the health facilities. We now discuss our FP data regularly and the service providers are able to identify gaps and strategies to improve performance. We are able to take FP services where they are most needed.
A data assistant in Uganda noted how more accurate facility-level data was used to better target services to the population, leading to improved access to and quality of services:
Most of the facilities are now prioritizing using data for decision-making, which is evidenced by graphs and charts drawn in some FP units supported by TCI. The division has observed a huge improvement in terms of FP service delivery access and quality. Through outreaches—a TCI evidence-based intervention—FP services are taken closer to the urban poor who may not afford transport to the facilities.
- Enhancing coordination of partners
The stakeholders’ stories, particularly from East Africa and Nigeria, illustrated how enhanced coordination leads to more agile, impactful programming. For example, a government official in Bauchi state, Nigeria, emphasized how TCI has empowered her to coordinate the various implementing partners:
TCI provided a platform for us to see a reason for all partners to come together under one umbrella so that we could move to make things happen better and quicker with less stress but greater impact.… Now the agency knows and coordinates activities of partners. There is synergy now… every partner has a shared vision.
Several stories touched upon how TCI brought a range of stakeholders together to roll out implementation of evidence-based interventions, resulting in the integration of family planning into other health programmes, such as maternal, newborn, and child health, ultimately meeting women’s and families’ needs more holistically. A health educator of Rivers state, Nigeria, explained:
We were able to build family planning into our other programs—Maternal Newborn and Child Health Week, for example… We will be having quarterly meetings at the local level to bring different stakeholders together through the help of TCI.… We are able to expand more.
Some stories illustrated how the coordination effort went beyond direct partners and donors and reached out to the local government units. In Senegal, a government official described how TCI activated coordination across three of the five municipalities comprising the region:
With TCI, our three city councils have pooled their funds to deliver FP and AYSRH services in some geographies. We will respond collectively… and this practice is unusual. All of the activities in the action plan are implemented at the same time in Bignona, Oussouye, and Ziguinchor.
Other stories pointed to the significance of enhanced coordination between the public health system and the private sector for improved family planning service delivery as a result of TCI’s capacity strengthening. A service provider in Uganda said:
There is better coordination with private health facilities and we have [an] improved referral system. So, a client who does not want to come to the public health facility will still get the family planning service at the private health facilities. Before this, we were only concentrating on the public facilities. We have strengthened the capacity of the midwives providing family planning in the private sector.
This coordination also extended to improving linkages between the community and service delivery system by activating community structures. For example, strengthening women’s groups was an evidence-based practice in India that aimed to strengthen community accountability and facilitate access to family planning services among the urban poor. A government official conducted a visit to a women’s group and noted his excitement at seeing the mother’s group successfully activated:
I was enthralled to hear this amazing true story of these women champions. If these women groups were to work as a true connection between the community aspirations and health service delivery, then the day will not be far when every citizen of the country will get quality health care services provided for them by the government.