We will conduct a two-armed cluster RCT to evaluate the effects of the intervention on the motivation, performance, and retention of VHTs in Uganda’s Masindi District. The intervention comprises of a recognition-based non-financial incentives package, designed in August 2021 in collaboration with local leadership from Masindi District; this process is described in further detail below.
Study Setting and Participants
The study will be implemented in Uganda’s Masindi District (see Fig. 1). As of July 2021, Masindi District has nine sub-counties, 32 parishes, and 312 villages. Masindi is a predominantly rural district, with most of the population living far from the main roads. Around 30% of the population lives on less than $1 per day, per a 2012 estimate (19). Compared to national averages, Masindi District has a higher maternal mortality rate (499 per 100,000 live births, as compared to 435 per 100,000) and a higher proportion of adolescent deliveries (24.8% compared to 20%) (20) compared to the national rates. Vaccination coverage is lower than the national average, with around 71.8% of children receiving the DPT3 vaccine on schedule, compared to 84% nationally (20). The District also experiences a high prevalence of malnutrition and infectious and communicable diseases (20). In 2020–2021, Masindi District was ranked 107th out of 136 Districts per a composite index calculated by Uganda’s MoH which ranks districts from best to worst-performing, signifying poor health sector performance (21). This index considers staffing levels; coverage (e.g., tuberculosis case notification rate, antenatal care 4th visit coverage, proportion of deliveries in health facilities, etc.); quality of care (e.g., proportion of maternal deaths that are reviewed, proportion of perinatal deaths that are reviewed, etc.); community health services (e.g., community VHT quarterly reporting rate, proportion of children under-five dewormed in last six months); and management (e.g., local government performance assessment score, supervision performance assessment and recognition strategy score) (21). Per this index, the national average is 64.4%; Masindi District is at 57.3% (21), which indicates that Masindi District is in the lower 50th percentile of districts for critical health and sanitation parameters. Masindi District’s selection as the study site was guided by the poor coverage of maternal and child health outcomes, and the topographical diversity within the three distinct ecological zones, which allows for assessment of the VHT program under a range of geographical conditions.
Inclusion Criteria
VHTs who send at least one report from their community registers to their supervisor using the designated quarterly reporting form (see Appendix 2) will be considered “active” and are eligible for enrollment in the study, following consent. As of 2020, there was an estimated 517 VHTs across 32 parishes in the district, with an average of 15 VHTs per parish. The study plans on recruiting all active VHTs.
Randomization and Blinding
The intervention will be randomized at the parish-level. On average, parishes comprise between five and nine villages, with each village having around two active VHTs, thus providing between 10–18 community members per cluster. Parishes will be randomly assigned with equal probability to either the intervention or comparison group within each sub-county. All 32 parishes will be randomized. Randomization will be conducted by a biostatistician independent to the study team through random number generation using STATA (22). The random numbers will be generated uniformly, then ordered ranks will be generated separately with a rank order of 0 or 1. Rank 0 will be labeled the comparison parish and Rank 1 will be labeled the intervention parish. Rank orders will then be randomly assigned to parishes. There will be 16 parishes/arm and approximately 304 VHTs in each arm.
The intervention will not be blinded to the study team as the study team is engaged in supporting the delivery of the intervention to support Masindi district. The intervention will not be blinded to the participants as the intervention involves public recognition of high performing VHTs.
The Intervention
The development of the intervention incentives package is guided by the insights gained through a prior study that used a discrete choice experiment to understand incentive preferences of the CHWs in Uganda (17). The study indicated that reliable transportation (such as bicycles) and recognition (through use of branded uniforms and/or identification cards) were highly valued by the CHWs. CHWs were willing to accept a decrease in salary of Ush 31,240 (US $8.5) for identity badges and of Ush 85,300 (US $23) for branded uniforms to no form of identification (17).
To determine the intervention to be tested and the criteria for assessing performance, the study team met with 16 stakeholders in Masindi District in August 2021. The stakeholders included VHT/Parish Coordinators; Health Assistants, who supervise VHTs; Health Inspectors; District Health Educators; District Health Officer; Biostatistician; and representatives/focal persons for district quality improvement, district surveillance, HIV, and malaria programs. The results of the prior study were presented to the stakeholders and based on considerations of sustainability of the intervention and the ability of the local district to financially sustain the intervention, the proposed intervention was adapted for the context. These discussions resulted in the following recognition-based incentives package: (1) a certificate which includes a name of the VHT and language describing the VHTs’ exemplary performance; (2) a branded jacket that would identify the VHT as a high performer (see Fig. 2); and a (3) a public recognition of VHTs who have been actively serving their communities.
All VHTs who have submitted their quarterly reporting forms (see Appendix 2) to the health facility will be considered for recognition. VHTs will be assigned to different strata based on the size of the rural/urban parishes (i.e., urban small, rural small, rural large) given the differences in the number of households, proximity of households. Finally, the top 10% of performers within each stratum will be publicly recognized. Performance of the VHTs will be assessed based on the following indicators: proportion of children under five years of age with up-to-date immunization, proportion of sick children under five years of age the VHT has attended to, and proportion of women who attended at least four antenatal care visits. These indicators were chosen as they are what VHTs record in and report from their registers and are likely to be achieved by VHTs. The denominators for each variable will be abstracted at the village-level from the official population projection statistics provided by the Masindi District government, District Planner’s Office.
The public recognition ceremony will be convened on a quarterly basis, as the VHTs provide their activity reports on a quarterly basis. Representatives from the district will convene and lead the ceremony and all VHTs from the sub-county will be invited.
Primary Study Outcomes
This study utilizes a mixed-methods approach to understand the impact of the intervention on the primary outcomes. The primary outcomes for this study are changes in the following: (i) VHT performance (i.e., number of home visits; proportion of children under five years of age with up-to-date immunizations; proportion of sick children under five years of age the VHT has attended to; and proportion of women who attended at least four antenatal care visits); (ii) VHT motivation (assessed using the validated Close-to-Close (CTC) Provider Motivational Indicator Scale (23); (iii) VHT retention (measured through monthly self-reports. Secondary outcomes include: trends in service delivery and trends in the adoption of sanitary practices. VHT performance is challenging to assess accurately as often VHTs may not have the tools to record their visits (e.g., registers), may not have the training to correctly record the information and may not maintain records to accurately recall their activities. Therefore, service delivery data will be collected in three specific ways: (1) through VHT surveys conducted at baseline and endline; (2) through the abstraction of administrative data from health facilities conducted every quarter as VHTs are required to report quarterly; and (3) through monthly phone surveys. See Table 1for the list of outcomes and the related data collection activities to measure them.
Table 1
Primary Outcomes and Related Data Collection Activities
Outcomes
|
Data Collection Activity
|
1. VHT Performance & Trends in Service Delivery
No. of total home visits in the last month / last 3 months / last 6 months
No. of visits for antenatal care
No. of visits for postnatal care
No of visits to support immunization of children
No. of referrals made to health facility
No. of sick under-5 children attended by a VHT
No. of people provided with HIV counseling
No. of people provided with TB counseling
No. of people provided with general counseling
|
VHT Survey
VHT Phone Survey
Health Facility Data Abstraction Activity
|
2. VHT Motivation
Measured through the CTC Provider Motivational Scale. This scale has four sub-modules on work satisfaction, organizational commitment, community commitment, and work conscientiousness.
|
VHT Survey
|
3. VHT Retention
Assessed through the number of names listed on the VHT roster
|
Primary: VHT Listing Activity; VHT Phone Survey
Secondary: VHT Survey
|
4. Trends in Adoption of Sanitary Practices
No. of households with latrines
No. of households with improved latrines
No. of households with handwashing facilities
No. of households with safe drinking water
No. of households that are open defecation free
|
Health Facility Data Abstraction Activity
VHT Survey
|
VHT Recruitment and Mobilization
The study team is working with the District to implement this study. Initial meetings were held to orient the District leadership to the study concept and study team. District leadership comprised of the District Health Officers (DHO), District Focal Person for Community Health Activities, and the Biostatistician. The District Focal Person provided the study team with an orientation on VHT activities and responsibilities, as well as a list of currently active VHTs and Parish Coordinators. The study team will utilize this list to reach out to VHTs for recruitment and consent.
Training Data Collectors
A three-day training will be held to orient data collectors to the study and provide them with training on the study tools, human subjects research and ethics, and the consenting process. The third day of training will focus on using Open Data Kit (ODK) (24), a tablet-based data collection platform, and on pretesting the survey tool. All data collectors will have received a Bachelor’s degree, and have fluency in Runyoro, Lugbara, and/or Kiswahili – the prevalent local languages in the study district.
Data Collection and Timeline
Trained data collectors will implement the data collection activities. See Fig. 3 for the timeline of the intervention and data collection activities.
VHT Survey: A structured questionnaire will be administered to all VHTs at baseline and endline. The survey comprises of the following modules: demographics, VHT experience (i.e., current engagement as a VHT, compensation, assets and other revenues, trainings, responsibilities), VHT performance (i.e., their activities and health services provided to the community over the last three months), VHT motivation, and COVID-19 experiences as a VHT. The VHT motivation module was adapted from the 12-item Close-to-Community Health Workers (CTC) Provider Motivational Indicator Scale and comprises of four factors: work satisfaction, organizational commitment, community commitment, and work conscientiousness(23). The COVID-19 module was developed by the study team. The survey will be implemented in three languages: Runyoro, Lugbara, and Kiswahili. Survey data will be digitally collected on tablets using ODK(24).
Health Facility Data Abstraction: The study team will abstract service delivery records from the paper reports that VHTs submitted to health facilities to triangulate VHT performance data and capture trends in delivery of health services. Health facility data will be abstracted from the HMIS VHT 097b: VHT/ICCM Quarterly Village Report (see Appendix 2). This will occur on a quarterly basis – at four points in time for the prior three months.
VHT Listing: The study team will work with the VHT coordinator and health facility in-charge to list out all active VHTs at baseline. This activity will also be used to gather and update contact details and phone numbers for the VHTs. This will occur at three timepoints: baseline, midpoint (six months), and endline and will serve to monitor trends in the retention of VHTs.
VHT Phone Survey: All VHTs who have a phone will be administered a brief monthly phone survey to report on key performance indicators over the last month. The VHTs will be called by trained research assistants who are stationed in a call center in Kampala, Uganda. Calls will be made in local languages over the course of four days. The research assistant will make at least three attempts each month to reach the VHT. If the phone number is no longer active or connected to the VHT, the study team will reach out to the Parish Coordinator to connect with those VHTs. The questions are adapted from questions included in the VHT performance module in the VHT Survey. This data will be compared against the quarterly-administered health facility data abstraction activity’s data to validate those data. The survey will be administered using ODK (24).
Focus Group Discussions (FGDs): FGDs will be conducted with VHTs and community leaders at baseline and endline. The goal of the FGDs with VHTs is to understand the type and structure of incentives that they prioritize. Community leaders will be engaged with the FGDs to provide an understanding of their relationship with VHTs as well as their perceptions of VHT activities in advancing health services. Between 9–10 FGDs will be held with VHTs and 9–10 with community leaders; there will be between 6–8 participants in each FGD. FGDs will continue until saturation is reached. Each FGD will have two trained facilitators: one will moderate the discussion and the other will take notes.
Sample Size Estimation
The sample size was calculated using the number of household visits completed per month as the outcome measure. Tweheyo et al. (2021) reported a mean of 55 household visits completed per month by VHTs for a 500 household catchment area (25). To be able to capture an increased mean in the number of household visits conducted per VHT by a meaningful 15% increment, the variance between two independent samples would be: \({\delta }^{2}=3.80\). The sample size was calculated using a pre-specified power of 80%, given by \({Z}_{1-\beta }=0.84\), for two independent samples (intervention and comparison) following a normal distribution, with a z-statistic of 1.96, at 95% confidence interval to detect a pre-specified clinical difference between study arms, of 15%, given by ∆ = 0.15. When there are an equal number of individuals per cluster – n=19 (pooled mean) VHTs per cluster – we use Eq. 1 (26,27) modified by Rutterford et al. (2015) to account for a fixed number of clusters, in this case the 16 clusters per study arm for a total of 32 clusters (28).
Substituting in Eq. 1 below, and using a conservative intra-class correlation co-efficient of 0.05 for lay workers (26, 27), gives us a sample size (m) of 19,199 households per trial arm. When adjusted for 10% non-responses, it is 21,110 new household visits per month for each trial arm. This indicates that for the 16 clusters with known cluster size n = 19 VHTs each (304 VHTs per arm), each VHT would achieve 69 new household visits per month.
Equation 1:\({m=\frac{{({Z}_{1- \alpha /2}+{Z}_{1-\beta })}^{2} {2\delta }^{2} }{{\varDelta }^{2}} [\left(1+(n-1\right)}_{\rho }]\)
In sum, by the end of the trial, the sample size is powered to detect 69 mean household visits per month in the intervention arm, as compared to a base mean of 55 household visits per month in the comparison arm.
Patient and Public Involvement
FGDs held with VHTs and community leaders as well as collaboration with District-level leadership informed the incentives package and intervention design. Study results will be disseminated in collaboration with District-level leadership and policy-level stakeholders to further inform VHT policy in Uganda.
Analysis Plan
To assess the impact on the primary outcomes, we will perform a linear regression analysis, adjusting for a cluster-effect, at the end of each quarter and the endline. Further, a difference-in-differences (DiD) analysis fitted using ordinary least square regression, including fixed effects at the parish level, will be conducted. The DiD approach can further confirm the effect of the intervention by controlling for baseline differences between the intervention and comparison group as well as temporal differences that may have resulted from underlying changes over time. Using fixed effects at the parish level will account for any unmeasured time-invariant factors that may influence uptake of the program. A per-protocol sub-group linear regression analysis adjusting for a cluster-effect will also be completed to examine the effect modification of recognition receipt status and social-economic status of VHTs on the outcome.
For the qualitative data collection (i.e., FGDs), transcripts will be reviewed by the study team. The study team will develop a codebook using a combination of deductive and inductive approaches to reflect the study’s objectives and address any emergent themes from the data. A team of two-to-three researchers will initially apply the codebook to two transcripts and review interrater reliability before moving forward to coding the remaining transcripts. The research team will meet regularly to discuss the coding process and ensure alignment and agreement. Data will be coded and analyzed to better understand the types and structures of incentives that motivate VHTs, the community’s relationship and engagement with VHTs, and community perceptions of VHT activities in advancing health, sanitation, and quality of care services. Data will be coded using QSR’s NVivo software (29).
Data Monitoring and Management
Members of the study team will be responsible for reviewing incoming data. The study team does not anticipate any harm to be caused to study participants because of the intervention. Data from the VHT survey and VHT phone survey will be collected using ODK, a secure data collection platform. Any paper forms collected in the study will be kept in locked and secure storage cabinets, with only requisite personnel to have access. Any data that has personally identifiable information will be kept on a secure server.