Study design
The study, scheduled to be conducted over three months during the flu season, will be a hybrid type II Effectiveness-Implemenation Trial, which will employ a multi-tiered randomized controlled trial (RCT) to evaluate the effectiveness of Rapid Verbal Persuasion (RVP) and incentives using both qualitative and quantitative methods. The design consists of two embedded trials (Fig. 1):
-
Two-Arm Randomized Controlled Trial: First, vaccination clinics will be randomly assigned in a 1:1 ratio to either a) the experiment group using RVP, or b) the standard practice group without RVP. Next, within the experimental group clinics, visitors will again be randomly assigned in a 1:1 ration to either a) the intervention group receiving RVP, or b) the control group not receiving RVP. This two-arm RCT will last for one month.
-
Cluster Randomized Factorial Trial: Following the two-arm RCT, all vaccination clinics in the experimental group will adopt RVP and be further assigned to one of 16 experimental conditions in the factorial trial. Each condition will have an equal number of clinics, receiving a specific combination of incentive strategies (Table 1). This cluster randomized factorial trial will run for two months. A factorial design is optimal, given the study's focus on multiple incentive forms. This approach allows for evaluating individual incentives and their combinations without proportionally increasing the sample size. Unlike traditional two-arm or multi-arm controlled trials, the factorial design can efficiently investigate potential interactions between different incentives, aligning well with the study's objectives and logistical constraints21.
It should be noted that throughout the entire study, vaccination clinics in the standard practice group will not implement RVP.
Study setting
The study will be conducted across four provinces in China, purposively selected to represent diverse socioeconomic, cultural, and policy environments:
-
Guangdong: a developed region;
-
Anhui and Sichuan: moderately developed regions;
-
Guizhou: an underdeveloped region.
This selection ensures a relatively representative sample of China's varied economic landscape and influenza vaccine policies. For instance, in Anhui, individuals must pay for out-of-pocket influenza vaccines, while certain districts in Guangdong offer free vaccination to specific groups (e.g., elderly, school-aged children) through government subsidies. Therefore, the purposive sampling of these provinces allows a better representation of the spectrum of China's economic development and the varied coverage of the influenza vaccine.
The study will be conducted in healthcare institutions with vaccination clinics that meet the following criteria: (1) hospitals at all levels, community health centers, community health stations, township health centers with vaccination clinics, and village clinics certified for immunizations. Exclusion criteria include (1) those who are engaged in other intervention studies aimed at increasing the influenza vaccine rate, (2) those who are involved in other implementation studies about incentives, and (3) those who are beneficiaries of subsidies from vaccine companies.
Participants and recruitment
Identification of participants
The target population for the RVP consists of vaccination clinics' visitors for inquiries and either receive vaccinations themselves or accompany others for vaccinations (for instance, parents bringing their children for routine childhood vaccinations). Individuals who proactively seek or have already received the influenza vaccine and those with contraindications will be excluded from the study.
The target population for incentives is vaccination staff at the vaccination clinics. The inclusion criteria for participants will be as follows: (1) Vaccination staff (including clinicians, public health physicians and nurses) working at the sampled vaccination clinic; (2) working for ≥ six months in vaccination clinics. The exclusion criteria will be as follows: (1) absence from duty for over a month for any reason; (2) informal staff, including those on attachment, rotation, or internship; (3) staff temporarily assigned to other tasks; (4) involvement in other interventions aimed at increasing influenza vaccine rates; (5) participation in other studies concerning incentives.
Recruitment of study settings and participants
The study will employ a multi-stage sampling approach to select vaccination clinics. Initially, the province of Guangdong representing developed regions of China, along with Sichuan and Anhui, symbolizing moderately developed areas, as well as Guizhou, exemplifying less developed areas, will be selected as study sites through purposive sampling. In the meantime, all healthcare institutions that meet the inclusion and exclusion criteria will form the sampling frame. In the second stage, the purposive sampling method will be further employed to select 32 health institutions with vaccination clinic from the sampling frame.
Vaccination clinics' visitors will be recruited in two phases. First, during the two-arm RCT, researchers will recruit visitors from sampled vaccination clinics in the experimental group during the initial month. Since participants will be randomly assigned to either the intervention or control group within each clinic, recruitment will occur before residents register or inquire for vaccination. At this point, researchers will explain the study's purpose and protocol to eligible participants, and only those who consent and sign the informed consent form will be recruited and subsequently enter the randomization process.
In addition, during the final two weeks of the study, researchers will visit all sampled vaccination clinics in both the experimental and standard practice groups to recruit visitors through cluster sampling. To ensure vaccination staff implement RVP naturally without researchers observation, recruitment will occur after visitors complete the entire vaccination process. At this point, researchers will explain the study's purpose and protocol to eligible participants. Only those who consent and sign the informed consent form will be recruited to complete a survey.
To recruit vaccination staff, researchers will initially employ cluster sampling to select eligible staff from sampled vaccination clinics. Prior to commencing the study, researchers will hold online conferences to explain the study's purpose and protocol to potential participants. Only individuals who consent to participate and sign the informed consent form will be recruited and proceed to the group randomization process.
Sample size calculation
The sample size for the first-phase two-arm RCT was calculated based on influenza vaccination status as the primary outcome. A prior RCT by our team (study pending publication) showed a 0.91% vaccination rate in the control group and 14.62% in the RVP group. We set the rate of type Ⅰ error ɑ at 0.05, the power (1-β) at 0.8. to and the sample size is 59 for each group, 118 in total. However, since RVP implementation is short-term and increasing recruitment would not raise intervention costs, we expanded the sample size to 800 to enhance statistical power, ensure good representativeness, and improve external validity.
To determine the sample size for evaluating real-world effectiveness of RVP in the final two weeks, we used influenza vaccination status as the primary outcome and the same parameters as in phase one, resulting in 118 residents. However, this phase involves cluster sampling from both RVP and standard practice clinics, which increases sample variance. Therefore, the design effect (Deff) was set at 2. Additionally, to account for potential dropout, we increased the sample size by 20%, resulting in a final total of 296 participants.
In the cluster randomized factorial trial, only 16 vaccination clinics were identified. Therefore, this protocol calculates the achievable statistical power for evaluating another primary outcome—whether vaccination staff provide the RVP. Initial surveys indicated that about 50% of staff provide health education on the influenza vaccine. Additionally, experts in implementation science, health economics, and immunization programs concurred that increasing the RVP implementation rate to 75% through incentives would be significant. Assuming the presence of 16 vaccination clinics with 5 vaccination staff per clinic, and the type I error rate ɑ at 0.05, the statistical power for this trial is calculated to be 0.7584.
Intervention development based on the co-production principle
Health intervention: Rapid verbal persuasion (RVP)
This study will employ the Rapid Verbal Persuasion (RVP) intervention, which has been developed and tested by our team previously. The RVP consists of a concise verbal script delivered by vaccination staff to visitors not initially seeking influenza vaccination. The development of the verbal script was theoretically guided by the Health Belief Model (HBM) and the 3C Model. The HBM posits that health behaviors are influenced by an individual's perceived susceptibility to and severity of a health condition and the perceived benefits and barriers to taking action22. The 3C Model, specifically developed for vaccine hesitancy, identifies Confidence (in the vaccine's safety and effectiveness), Complacency (about the risk of influenza), and Convenience (of getting vaccinated)23. We further refined the scripts using empirical data from a scoping review, team discussions, and stakeholder interviews.
Finally, each developed script contains a core component and variable components. The core component, "If you wish, you can conveniently get a shot today" serves as a cue to action (in line with the HBM) and emphasizes convenience (reflecting the 3C Model). The variable components consist of multiple key points, which are tailored to address specific barriers based on visitors profiles (e.g., children, adults, elderly), targeting confidence and complacency (3C Model) while also addressing perceived benefits and barriers (HBM). For instance, the script for an adult might emphasize the benefits of vaccination for personal health and protecting vulnerable family members. An example variable component for young parents accompanying their children to the vaccination clinic read: "Receiving the influenza vaccine can reduce the likelihood of colds you catch, and even if you do get sick, the symptoms will be much milder (point 1), so it won't affect your work (point 2); You are young and may likely recover quickly after flu, but you may transmit it to your children or seniors in your home, which could lead to pneumonia and other severe consequences (point 3)." The specific verbal scripts for different populations could be found in Supplementary Table 1.
The intervention will be triggered in two scenarios. First, when visitors request non-influenza vaccination prescriptions or inquire about non-influenza vaccinations at the registration desk, the vaccination staff responsible for registration will deliver the RVP. Next, when visitors receive non-influenza vaccinations at the injection station, the vaccination staff responsible for injection will deliver the RVP. During the implementation process, vaccination staff will be required to strictly adhere to the core component of script but can adapt variable components to match visitors' demographics and language preferences,and ensuring at least one key point is covered. This approach ensures effective communication while allowing flexibility in delivery. All descriptions of RVP align with the Template for Intervention Description and Replication (TIDieR) checklist (Supplementary Table 3) 24.
Implementation strategy: Incentive-based implementation strategies
The incentive-based implementation strategies were developed through a four-step co-production process guided by co-design and stakeholder engagement25. First, a literature review was conducted to identify potential incentives used in previous studies to promote implementing evidence-based practices (EBPs). Next, we interviewed a multidisciplinary team of immunization program experts, healthcare administrators, and frontline vaccination staff to gain their perspectives on adapting those incentives to promote RVP in the Chinese vaccination context. Subsequently, a Delphi study was conducted with a panel of experts in immunization programs, public health, implementation science, and healthcare systems to further refine the list of incentives and ascertain their corresponding levels. Finally, a Best-Worst Scaling (BWS) survey was conducted among vaccination staff and institutional administrators to elicit their preferences regarding the different types and levels of incentives26.
Based on the findings from this co-production process, four types of incentives were selected for inclusion in the factorial trial. These four incentives, each with two levels (presence or absence), form the 16 distinct experimental conditions in the 2x2x2x2 factorial design (Table 1). Each incentive's specific content and delivery mechanisms are described in detail according to the Proctor Implementation Strategy Description Framework (Table 2)27. In addition, following the nine principles from behavioral economics13, we have developed a logic model to elucidate the mechanisms through which incentives may enhance the implementation of RVP (Fig. 2).
Table 2
Outline of intervention groups with four different incentive-based interventions
| | Interventions | Action | Actor | Target | Temporality | Dose |
| 1 | Flat upfront compensation | A one-time research compensation will be distributed to the vaccination clinic regardless of performance targets, and 200 RMB per person will be allocated to each staff of the vaccination clinic | The research team | Staff in the vaccination clinic which are responsible for pre-screening, registration, and vaccination | Compensation will be distributed in the first week of incentives implementation, but the purpose of compensation will be informed in advance | During the study period, the compensation will be disbursed once |
| 2 | Performance bonus | A one-time performance bonus will be distributed to the vaccination clinic that meets the performance targets, and 300 RMB per person will be allocated to each staff of the vaccination clinic with the specific amount to be determined by the department leader | The research team | Staff in the vaccination clinic which are responsible for pre-screening, registration, and vaccination | Bonus will be distributed at the the end of study, but the purpose and distribution mechanism of bonus will be informed in advance | During the study period, the bonus will be disbursed once |
| 3 | Focused discussions | During the research process, the research team, in collaboration with the leaders of vaccination clinics, will organize the focused discussion for all staff. This is to facilitate the sharing and exchange of brief verbal interventions and to optimize the delivery form of these interventions during implementation | The research team collaborated with the leaders of vaccination clinic | Staff in the vaccination clinic which are responsible for pre-screening, registration, and vaccination | Focused discussions will be organized in conjunction with departmental meetings in the middle of each month | During the study period, the focused discussion will be organized twice |
| 4 | Public Recognition | The institutional leadership will announce the implementation status of RVP and publicly recognition the vaccination clinic and staff in the WeChat work group for their contributions in health education, and encourage other departments to learn from the vaccination clinic | The institutional leadership | The vaccination clinic and its staff which are responsible for pre - screening, registration, and vaccination | Public recognition will be implemented at the end of study, but the purpose of public recognition will be informed in advance | During the study period, the public recognition will be implemented once |
| Note: The above incentive interventions are outlined in detail according to Proctor Implementation Strategy Description Framework. |
Randomized group assignment and blinding
Stratified randomization will be used to assign the 32 participating vaccination clinics to either the intervention group (with RVP) or the control group (without RVP) in a 1:1 ratio. The strata will be defined based on geographic location (rural/urban) and the level of healthcare institutions. Within each stratum, vaccination clinics will be randomly assigned to either group using the block_ra() function of R package randomizr designed for randomization.
The 16 vaccination clinics assigned to the RVP group will then be further randomized into one of 16 experimental conditions through complete random assignment. Sixteen unique random numbers will be generated using simple() function of R software and assigned to 16 vaccination clinics. Subsequently, these clinics will be allocated to different experimental conditions according to the order of the assigned numbers. All vaccination staff within the same clinic will be exposed to corresponding incentive condition. An independent statistician will conduct the randomization.
Due to the nature of the interventions, blinding the participating institutions and vaccination staff to their assigned interventions is not feasible. outcome assessors and data analysts will be blinded to the group assignment status.
Outcomes and measurement
This study employs the RE-AIM framework to comprehensively evaluate the effectiveness of RVP in boosting influenza vaccination rates and to assess the relative effects of various incentive strategies on facilitating the implementation of RVP among vaccination staff28. The specific outcomes include:
Primary outcome: This study includes two primary outcomes: fidelity of core component implementation of RVP and influenza vaccination status. First, the fidelity of core component implementation of RVP assesses whether vaccination staff have mentioned the core component of RVP as planned, measured as a binary variable during the final two weeks of the intervention period through exit interviews with visitors to vaccination clinics (Table 3). To mitigate the potential Hawthorne effect, investigators will conduct unannounced visits to each participating institution without any prior notification. The outcome will be recorded as '1' if at least 70% of eligible visitors confirm the staff referred the core component, and '0' otherwise. Influenza vaccination status will also be determined through exit interviews, with a '1' recorded if respondents confirm they received the vaccine, and '0' if not.
Table 3
Study outcomes of our study
| Outcome Type | Outcome Name | Outcome Explanation | Theoretical basis | Variable Type | Data Collection Tool | Data Collection Method | Data Collection Period | Outcome Measure Object |
| Primary Outcome | Fidelity of core component implementation of RVP | Whether vaccination staff have mentioned the core component of RVP as planned | RE-AIM: "Implementation" | Binary variable (0,1) | Questionnaire | Exit interviews with visitors | In the final two weeks of intervention period | Vaccination staff |
| Primary Outcome | Influenza vaccination status | Whether the eligible visitors who participated in exit interviews have received the influenza vaccine | RE-AIM: "Effectiveness" | Binary variable (0,1) | Questionnaire | Exit interviews with visitors | In the final two weeks of intervention period | Vaccination clinic visitors |
| Secondary Outcome | Fidelity of comprehensive implementation of RVP | Whether vaccination staff have mentioned both the core and variable components of RVP | RE-AIM: "Implementation" | Binary variable (0,1) | Questionnaire | Exit interviews with visitors | In the final two weeks of intervention period | Vaccination staff |
| Secondary Outcome | Acceptability of RVP | Whether vaccination staff consider the implementation of RVP for influenza vaccine in their daily work is acceptability. | RE-AIM: "Implementation" | Continuous variable | TFA questionnaire | Questionnaire Survey with vaccination staff | Within two months following the end of project | Vaccination staff |
| Secondary Outcome | Adaption of RVP | Adaptations in the Verbal Scripts during the intervention period | RE-AIM: "Implementation" | Text variable | FRAME questionnaire | Questionnaire Survey with vaccination staff | Within two months following the end of project | Vaccination staff & vaccination clinic visitors |
| Secondary Outcome | Cost | The cost of conducting the research depends on the costs related to RVP, the incentives utilized and downstream costs | RE-AIM: "Implementation" | Continuous variable (RMB) | Expenditure statement | N/A | Within two months following the end of project | N/A |
| Secondary Outcome | Sustainability of RVP | The probability that the institutions will routinely adopt rapid verbal persuasion following the project's conclusion | RE-AIM: "Maintenance" | Text variable | Interview framework based on NPT | Qualitative interviews with vaccination staff | Within two months following the end of project | Vaccination staff |
| Secondary Outcome | Sustainment of RVP | The status of vaccination staff at participating institutions in continuing the implementation of RVP after the end of project. | RE-AIM: "Maintenance" | Binary variable (0,1) | Questionnaire | Telephone Follow-up with vaccination staff | Within the seventh month following the end of project | Vaccination staff |
| Notes: TFA: the Theoretical Framework of Acceptability; RVP: Rapid Verbal Persuasion; NPT: Normalization Process Theory; FRAME: Framework for Reporting Adaptations and Modifications-Enhanced. |
Secondary outcomes: This study evaluates a spectrum of secondary outcomes associated with the implementation of RVP by vaccination staff, including fidelity of comprehensive implementation of RVP, cost, acceptability, adaptation, sustainability and sustainment. Firstly, unlike the primary outcome, fidelity of comprehensive implementation of RVP will be recorded as '1' if at least 70% of eligible visitors confirm the staff mentioned both the core and variable components of RVP, and '0' if not. Costs will be quantified based on the comprehensive financial records maintained throughout the study, including implementation costs related to incentives, intervention costs for the RVP, and downstream costs. Acceptability and adaptation to the implementation of RVP will be gauged via a questionnaire survey. Acceptability of RVP will be measured using a survey questionnaire based on the Theoretical Framework of Acceptability (TFA)29, while adaptation of RVP will be captured through questionnaire based on the the framework for reporting adaptations and modifications-enhanced (FRAME)30. These measures will be assessed within two months after the study concludes. Sustainability pertains to the probability that the institutions will routinely adopt rapid verbal persuasion following the project's conclusion. This variable will be measured within two months after the study concludes using in-depth interviews guided by the Normalization Process Theory (NPT)31. Sustainment pertains to the continuation of RVP implementation by the vaccination staff at participating institutions after the conclusion of the study. This outcome will be collected through telephone follow-ups in the seventh month following the end of the project. Notably, this project did not incorporate outcome measures related to the Reach dimension of the RE-AIM framework. This omission was based on the rationale that both Reach and Fidelity assess the implementation of RVP by vaccination staff, making a separate Reach measure unnecessary.
Statistical methods
Mean ± standard deviation (SD) or median (25th − 75th percentile) will be expressed for continuous variables, and count (percentage) is used to describe categorical variables. Additionally, secondary outcomes including cost, acceptability and sustainability will also be reported as mean ± SD, and adaptation will be described through qualitative texts.
In the investigation of the effects stemming from incentives, our analytical approach unfolded sequentially. Firstly, we encoded the impact of each level of experiment conditions as -1 and + 1, aiming to establish a balanced design for subsequent analysis. Secondly, both crude and adjusted risk ratio (RR) values, along with 95% confidence intervals (CIs) were calculated to assess the main effect and interaction of four distinct incentives on fidelity using modified Poisson regression models. These models would be adjusted for a range of demographic and professional characteristics of vaccination staff, including sex at birth, age, education level, professional title, job responsibilities, professional identity, years of work experience, income level and employment province. Furthermore, to account for the impact of institution characteristics on fidelity, the generalized linear mixed-effect model (GLMM) would be further employed. This model will estimate the odds ratio (OR) and 95% CIs, by adjusting for institution characteristics (e.g., the type, location and nature of institution, the form and periodicity of vaccination services) and the previously mentioned characteristics of the vaccination staff. Within this model, the institution characteristics are designated as level-2 factors, while the vaccination staff characteristics are categorized as level-1 factors.
For the analysis of influenza vaccination status, two statistical models would be conducted. In the first phase of the two-arm RCT, visitors are randomized within each cluster without the need to account for intra-class correlation (ICC). Therefore, modified Poisson regression models would be used to to examine the association between RVP and vaccination status, estimating RRs along with 95% CIs. The multivariate model will incorporate visitors' characteristics including sex at birth, age, education level, occupation, residence, and knowledge and attitudes towards influenza vaccination. However, when reassessing the effectiveness of RVP after the entire study concludes, ICC must be considered as data will be collected from clinics that fully implement or do not implement the RVP. At this stage, a generalized linear mixed-effects model (GLMM) will be employed to estimate ORs and 95% CIs, adjusting for institutional characteristics as two-level covariates and visitor characteristics as one-level covariates. The institutional and visitors' characteristics adjusted for will remain the same. Lastly, the sensitivity analyses will be executed by imputing missing data using random forest models to enhance the reliability of our findings. Data management and analyses will be performed using R statistical software (v. 4.1.2) and SAS 9.4 (SAS Institute Inc., Cary, NC) with a significance level of 0.05 (two-sided).
Data monitoring and quality control
Given the minimal risks associated with participation in this study, which do not exceed everyday life risks, the establishment of a formal Data Monitoring Committee has not been deemed necessary. Nonetheless, the study will undergo continuous oversight, encompassing progress monitoring, data quality, and integrity assessments, as conducted by the Ethics Review Committee of Southern Medical University.
All data management and field data capturing in our study will be conducted using the REDCap platform32. The data collection system will leverage REDCap's functions to design stringent logical progressions and conditional constraints in order to enhance the accuracy and efficiency of questionnaire completion. Data collectors will be subjected to two rounds of standardized online training after recruitment to ensure adept use of the data collection system and uniform comprehension of the collected item content. Furthermore, all data collectors will participate in an online exam, with only those passing the exams proceeding to the formal data collection stage. Finally, the quality control team will meticulously review each entry in the datasets to prevent any instances of incorrect, omitted, or misplaced data.