Study design
The proposed study is designed as a type III hybrid implementation trial, concentrating on the development and assessment of the effectiveness of a specific implementation strategy to enhance the adoption of BISC. Employing a mixed-method design, the study integrates quantitative and qualitative methods for data collection and analyses in a complementary manner. Furthermore, the research adheres to the principles of the Multiphase Optimization Strategy (MOST)(18, 19), incorporating a factorial randomized trial to ascertain the optimal set of implementation techniques within predetermined constraints (Fig. 1). Factorial designs facilitate the efficient exploration of multiple factors and their interactions, resulting in broader and more applicable conclusions. Moreover, they afford enhanced control over error variance, diminish sampling error, and can offer greater economy and flexibility compared to individual experiments for each factor.
The development of this study also adheres to PEDALS(20), a procedure our group has devised to steer the design of implementation research. PEDALS initiates by identifying real-world "Problems" in healthcare services and proceeds to identify EBP that can address these issues. Subsequently, it pinpoints "Determinants" to the implementation of this EBP. Following this, "Action" (an implementation strategy) is undertaken to address these implementation determinants, thus integrating the EBP into routine practice over the "Long-term." The "S" in PEDALS stands for "scale," involving the measurement of implementation effectiveness using appropriate designs and methods. Figure 2 provides a concise overview of each step in PEDALS for this study.
P EDALS – Problems to be addressed
As outlined, tobacco use presents a noteworthy challenge for the Chinese populace. Our study elects to concentrate on this matter owing to the considerable advantages that can be attained by tackling it and the viability of implementing effective programs to address it.
P E DALS – Evidence Based Practice to address the tobacco problem
A fundamental distinction between implementation research and traditional health services research lies in the approach to addressing identified issues. Instead of immediately devising new interventions, the focus shifts towards exploring existing EBP that could effectively tackle the problem at hand. In this context, the Brief verbal Intervention for Smoking Cessation (BISC) has proven effective in reducing tobacco use in numerous prior studies. However, this EBP remains untapped within China's primary care settings.
PE D ALS – Identifying Determinants (Barriers and facilitators) to the implementation for BISC
In this step, we employ a combination of literature review and stakeholder consultation to identify and prioritize determinants to implementation, proceeding as follows:
Literature review to identify barriers
We have conducted a scoping review to pinpoint barriers and facilitators to implementing BISC. We utilized the Consolidated Framework for Implementation Research (CFIR) domains and constructs to steer the data extraction process. A total of 142 studies published between 1996 and 2023 were ultimately analyzed and summarized. The predominant facilitators identified included knowledge, skills, and positive beliefs about BISC (individual characteristics), as well as training interventions (inner setting). Conversely, the primary barriers identified encompassed a lack of skills for effective BISC implementation (individual characteristics), insufficient time for implementation (intervention characteristics), and inadequate training in BISC implementation (inner setting).
Stakeholder consultation to prioritize barriers to be addressed
We will convene a consultative meeting with stakeholders (further details available on Engage stakeholders in co-production) to deliberate on the barriers identified earlier. This will enable us to refine the list by considering the specific context of our study, adding or removing barriers as necessary. Conference participants will also be tasked with rating and prioritizing the barriers for addressing, based on their potential impact on implementation outcomes and the feasibility of their resolution.
PED A LS – Taking Actions (implementation strategy) to tackle the implementation determinants
This study defines an implementation technique as a precise method or approach crafted to amplify the adoption, implementation, and sustainability of a particular intervention or program. These techniques are clear-cut components aimed directly at addressing specific barriers to change within a given context. In contrast, we define an implementation strategy as a comprehensive 'package' comprising multiple implementation techniques. Tailored to the context, this package is designed to synergistically achieve the desired implementation outcomes. Initially, we will compile a set of potential implementation techniques, which will then undergo refinement and optimization in the optimization trial.
Barrier-implementation technique matching
Once we have derived a prioritized list of barriers in the preceding step, we will employ the CFIR-ERIC (Expert Recommendations for Implementing Change compilation/ERIC) Matching Tool (21) to produce a set of initial implementation techniques aimed at addressing the identified implementation barriers. Our test matching process has yielded four potential primary implementation techniques for illustrative purposes, including conducting educational meetings, assessing readiness and identifying barriers and facilitators, identifying and preparing champions, and alter incentive/allowance structures (Supplementary Table S1).
We will construct an optimization conceptual model for the implementation of BISC, employing mixed methods guided by pertinent implementation frameworks. This conceptual model will integrate elements and their respective levels of implementation techniques and mechanisms that influence the implementation of BISC. We have developed a preliminary conceptual model based on our current understanding (see Fig. 3). This model will undergo further refinement through the following process.
Prioritization of Implementation Techniques
We will administer a Best-Worst Scaling (BWS) online survey to primary healthcare providers using the matched list of implementation techniques. This survey will enable us to prioritize the implementation techniques based on their potential impact and feasibility. The outcomes of this prioritization will inform the selection of candidate functions, which constitute the components of the implementation strategies.
Engage stakeholders in co-production
The study will subsequently involve stakeholders in identifying the specific forms (representing levels of implementation techniques), adhering to the principle of co-production. Participating stakeholders will encompass managers/providers and researchers specializing in implementation science, primary health services, and behavioral science. Discussions with stakeholders will also guide the determination of
prioritization of barriers, implementation techniques that should be consistent across all sites and those that are deemed optional for testing in the factorial trial. The selection of constant components typically relies on ethical or logistical considerations, ensuring that specific intervention elements are uniformly delivered to all participants. Additionally, caution is warranted when considering constant components, particularly in situations where resources are limited for examining all potential components that could be integrated into an optimized intervention.
Sample size for saturation
A review of qualitative research saturation assessments revealed that studies typically maintain relatively consistent saturation sample sizes, with empirical data indicating saturation within a narrow range of interviews (9–17) or focus group discussions (4–8) (22). Consequently, we plan to invite a minimum of 4 experts to participate in focus group discussions, with interviews concluding once information saturation is achieved.
The sample size for the BWS online survey may fluctuate based on various factors, including the research question, the number of attributes under evaluation, and the desired level of precision. Nonetheless, it is generally advisable to aim for a sample size of at least 200 respondents as a rule of thumb.
PEDA L S – Long-term use of BISC
The purpose of the study is to integrate BISC into routine practice for long-term use. Therefore, this study will assess a range of implementation outcomes that can collectively indicate the likelihood of achieving sustained BISC implementation over time. Implementation outcomes measurement adheres to the principles outlined in Proctor's Implementation Outcomes Framework (IOF) and the RE-AIM framework. There is considerable overlap between the RE-AIM and IOF across most categories (23). Table 1 provides a summary of the definitions and operationalization of these outcome measures.
Table 1
| Outcome | Variable | Theoretical basis | Explain | Type | Tool | Acquisition methods |
| IOF | RE-AIM |
| Primary outcome | Delivery of BISC | Fidelity | Implementation | The extent to which BISC was implemented in accordance with national clinical cessation guidelines, including asking if, when and how much smoke, talking about hazards of smoking, asking to quit smoking, making referrals and offering a quit line. | Quantitative data, dichotomous variable (0,1) | Quality assessment checklist | USP visit |
| Secondary outcome | Coverage of BISC Delivery | Penetration | Reach, Implementation | Percentage of providers who deliver BISC as compared to who have received BISC training or are expected to deliver BISC under defined conditions. | Quantitative data, percent (%, 0 ~ 100%) | Quality assessment checklist | USP visits |
| Other outcome | Applicability of BISC Delivery | Feasibility | Reach, Adoption, Maintenance | The extent to BISC can be successfully carried out within PHC provider. | Quantitative data, 0 ~ 100 points | NPT questionnaire | Questionnaire survey |
| Acceptance of BISC Delivery | Acceptability | Adoption, Maintenance | The perception among the PHC facilities that BISC implementation techniques is agreeable, palatable, or satisfactory. | Qualitative data | Interview outline | Semi-structured interview |
| Adoption | Adoption | The intention, initial decision, or action to try or employ BISC from the perspective of the PHC facilities. |
| Appropriateness | Adoption, Maintenance | The perceived fit, relevance, or compatibility of BISC for the PHC facilities; and/or perceived fit of BISC to address a particular issue or problem. |
| Sustainability | Maintenance | The extent to BISC is maintained or institutionalized within PHC provider's ongoing, stable operations. |
| Cost of BISC Delivery | Implementation Cost | Implementation | The cost of implementation, depends upon the costs of the BISC implementation techniques. | Quantitative data, | Administrative data | |
| Note: Outcomes will be obtained 2 months after the end of the optimization factorial trial. |
Primary outcome
The primary outcome will be the "delivery of BISC", indicating whether the provider administers BISC or not. The outcome will be assessed by Unannounced Standardized Patients (USP) - trained individuals who portray patients with specific medical histories, symptoms, and emotional characteristics, enabling the assessment of clinicians' attitudes (24), skills (25), and behaviors (26) in real practice. By conducting unannounced visits to clinicians, USPs mitigate the potential for observation bias, such as the Hawthorne effect (27), and account for variations in "case mix" due to their standardized presentation (28), thus offering a more valid, objective and precise measurement of implementation effects. We will employ three USP cases involving male current smokers with hypertension, type 2 diabetes, and cold, all requiring clinicians to provide BISC per national clinical practice guideline (Supplementary Figure S2). Previously, our study developed, validated, and implemented 11 USP cases (29) to evaluate the quality of PHC services, which contain a quality assessment checklist for the correct disposition of the physician in the desk design (Supplementary Table S2). These cases are reflective of the most prevalent conditions encountered in the PHC provider.
Secondary implementation outcomes
Those encompass the feasibility of BISC delivery, the level of acceptance toward BISC delivery, and associated costs, including those related to BISC itself and implementation strategies, as outlined in Table 1. Surveys will be administered to all participating clinicians and administrators to gather quantitative outcome data. The survey will consist of 20 items derived from the Normalization Process Theory (NPT) (Supplementary Table S3), with each item rated on a 5-point Likert scale ranging from 'strongly agree' to 'strongly disagree'. It's important to highlight that item 9 will be scored in reverse to accurately capture the degrees of agreement, neutrality, or disagreement. Total scores will vary between 20 and 100, with higher scores indicative of enhanced BISC implementation. In addition to outcome data, the provider survey will collect demographic information such as gender, age, education, job title, practice setting, discipline, working hours, number of implementation techniques utilized, and smoking status.
To complement the survey data, semi-structured interviews guided by NPT will explore (1) perceptions of BISC within medical facilities, (2) factors influencing the decision to adopt or implement BISC, (3) the perceived relevance and compatibility of BISC, and (4) the sustainability of BISC integration into ongoing operations. Face-to-face interviews will be conducted with purposively selected program participants, with the sample size determined by data saturation. Rapid qualitative analysis will be used featuring templated data capture and framework-guided analysis in lieu of traditional transcribing and coding.
PEDAL S – Scale for Evaluative Designs and Methodologies (Factorial trial)
In this section, we describe our study design and analysis plan to scale (measure) the outcomes of this optimization trial.
Determine optimization objectives
Optimization objectives are categorized as implementation outcomes (goals targeted by the implementation techniques) and constraints (boundaries during application of those techniques). The aim is to identify the most effective implementation strategy under specified constraints, balancing optimal outcomes with established constraints rather than solely maximizing outcomes.
Barriers to BISC implementation will be identified through a stakeholder focus group with physicians, nurses, and administrators. Non-modifiable or challenging-to-be modified barriers will be translated into optimization constraints, such as resource limitations on cost or provider time for implementation.
Study setting
The study will target primary care setting in China. Primary care in China including outpatient care in internal medicine departments at hospitals, as well as primary care doctors at specific PHC institutions. In urban areas, these PHC institutions include community health centers and community health stations/clinics in urban areas, and township health centers and village clinics in rural areas.
Study population
The study population will encompass licensed physicians, licensed assistant physicians, certified village doctors, and village sanitarians at the above PHC providers. Village clinics can be staffed by both licensed physicians or certified village doctors and sanitarians. Certified village doctors possess practice privileges limited to their clinic at the village level, even without a medical license, while village sanitarians are expected to operate under the supervision of the village doctor. The study will exclude any providers who have undergone training or taken part in trials related to smoking cessation.
Sample Size Calculation
We expect the proportion of clincians devleiring BISC The dichotomous variable "delivery of BISC" will serve as the primary outcome. Our earlier ACACIA study using USP to assess quality in PHC found the average level of BISC implementation by PHC providers at 15%. Out stakeholder consultation has determined that a 75% of the changes could be considered clinically meaningful. Assuming a two-tailed test, the alpha level of 0.05 and the desired power of 0.8, according to the formula below(30), we can get the sample size of one experiment condition\({ N}_{srs condition}(simple random sampling, srs)\):
\({N}_{srs condition}=\frac{{\left({Z}_{1-\alpha /2}+{Z}_{1-\beta }\right)}^{2}}{{2}^{k}{\left[arcsin\left(\sqrt{\frac{p+p\delta }{2}}\right)-arcsin\left(\sqrt{\frac{p-p\delta }{2}}\right)\right]}^{2}}\approx\) 9
Where \(p=0.30, {Z}_{1-\alpha /2}=1.96, {Z}_{1-\beta }=0.84, k=3, \delta =0.75\)
Therefore, approximately 72 samples are required to detect this effect (9 in each of the 8 conditions).
To avoid contamination, this study will consider conducting a between-clusters randomization approach, in which clusters (clinics or hospitals) are assigned as whole units for implementing experimental conditions. The clusters increase the variance of samples, which requires consideration of the design effect \({D}_{eff}=2\). Finally, the sample size of one condition (\({N}_{complex}\)) can be updated as follows:
\({N}_{complex}={N}_{srs conditions}\times {D}_{eff}=\) 144
Therefore, 144 samples are required (approximately 18 in each of 8 conditions).
Sampling procedure
The sampling frame will comprise providers who meet these criteria at facilities offering primary healthcare services. The study encompasses a range of facilities, including hospitals, community health centers, and stations in urban areas, as well as township health centers and village clinics in rural areas.
The sample will be chosen through a multistage, clustered sample design, encompassing all eligible clinicians (Fig. 4). The initial phase has been completed. In the first stage, we employed the purposive sampling, based on the geographical division and economy of China into Eastern, Central, and Western regions, four provinces were selected: Xinjiang, Anhui, Hunan, and Guangdong (Supplementary Figure S1). Subsequently, two cities were selected from Xinjiang and Guangdong, one city was selected each from Anhui and Hunan. In the second stage, local health authorities in the six cities were recruited that met the inclusion and exclusion criteria and were willing to participate. Considering three implementation techniques under evaluation (further details available on Sample size calculation for factorial trial), all facilities were regrouped into eight clusters according to a minimum sample size for each condition (n = 36). In the final stage, a minimum of 36 USP visits will be assigned to each experiment condition.
Group assignment and blinding
We will employ a factorial design, specifically a 2x2x2 factorial design, resulting in four conditions. Each condition will comprise a combination of the two 2-level implementation techniques. Completely random assignment of each primary sampling unit (PSU) to one of these four conditions will be conducted. The implementation techniques identified as constant components by experts will be allocated to all sites (refer to Table 2). This allocation is crucial to minimize potential contamination effects and will be completed within a two-month time frame. Blinding will not be feasible for the researchers and study subjects (institutions and providers) as both groups will actively participate in implementing the assigned interventions based on their allocation status. However, to mitigate subjective analysis, outcome assessors and data analysts will be blinded. Upon completion of all USP visits, the research team will disclose the participants' allocated interventions. Institutions will be decoded before analysis, and the allocation status will not be revealed to the data analysts throughout the trial.
Table 2
Hypothetical experiment with constant component (2x2x2)
| Experiment conditions | Constant component | Optional components to be tested |
| Assess for readiness and identify barriers and facilitators | Identify and prepare champions | Alter incentive/allowance structures | Develop educational materials |
| 1 | Yes | | | |
| 2 | Yes | Yes | | |
| 3 | Yes | Yes | Yes | |
| 4 | Yes | Yes | | Yes |
| 5 | Yes | | Yes | |
| 6 | Yes | | Yes | Yes |
| 7 | Yes | | | Yes |
| 8 | Yes | Yes | Yes | Yes |
| Note: a Yes means included in intervention; no means not included in intervention. The specifics form (such as "ranking PHC providers execution of BISC and providing spiritual rewards for excellence" ) of each component (such as "Alter incentive/allowance structures") will be developed by inviting stakeholders to work together, following the principles of co-production. |
Following multistage and clustered sampling, the allocation sequence will be generated using a random number table. Simple randomization for each group will be conducted independently by a statistician not affiliated with the research team. The randomization scheme will be sealed in an envelope and opened only at the time of formal randomization, enabling the researcher to execute the random assignment.
Data analysis
Data analysis in qualitative interview
In this study, interviews conducted to assess outcomes will be de-identified and imported into Nvivo 11.0 for organization, coding, and content analysis post transcription. Our qualitative data analysis will employ a combined deductive and inductive approach, facilitating the identification of established themes, patterns, and new discoveries(31, 32). The analysis will begin by using a deductive approach to identify key themes and patterns that align with the domains and constructs of the CIFR and NPT. Following this, an inductive approach will be utilized to explore and identify emerging themes or patterns within the data. To ensure inter- and intra-coder reliability, each interview will be independently coded by two team members simultaneously. Any discrepancies in coding will be addressed through discussion and resolution with the research team leader until a consensus on the final classification is achieved. Subsequently, all completed codes will be summarized and screened to identify determinants.
Assessment of main and interaction effect
Data will undergo de-identification prior to analysis. Baseline characteristics of participants will be presented, utilizing mean and standard deviation for continuous variables and frequency and percentage for categorical variables. The trial outcome analysis will adhere to the intent-to-treat principle. Modified Poisson regression models will be employed to calculate risk ratios (RRs) and corresponding 95% confidence intervals (CIs) to assess the main and interaction effects of the four implementation techniques on adherence to BISC practices among primary healthcare providers. The model will account for robust predictors of outcomes, including baseline adherence scores and socio-demographic factors of providers (33–35). Effect coding will be applied to each experimental condition, and subgroup analyses based on age, gender, provider level, and facility will be conducted to further explore the impact of different technique combinations on BISC implementation. Sensitivity analyses, involving imputation of missing values through random forest, will be performed to validate the robustness of the findings (36). Furthermore, Modified Poisson Regression Models will be employed to examine main and interaction effects on other continuous outcomes, adjusting for the same covariates as in the adherence analysis. All statistical analyses will be executed using SAS 9.4 (SAS Institute Inc., Cary, NC, USA) alongside the "mice" package in R 4.3.1 for data imputation (van Buuren S. Package 'mice'. Vienna: Comprehensive R Archive Network; 2015). A two-tailed P value less than 0.05 will be deemed statistically significant.
Systematic decision-making process
The decision-making process, guided by data from a factorial screening experiment (37), relies on the capabilities of factorial design to estimate main effects and interaction effects. These factors are integral to the systematic decision-making procedure, which includes: (1) Identify effective implementation techniques by leveraging the estimated main effects to pinpoint those that exhibit a statistically significant positive impact on the fidelity of implementing BISC. (2) Examine interaction effects to glean insights into potential synergies or antagonisms between implementation techniques. (3) Optimize implementation strategies by integrating information from both main effects and interaction effects. Researchers will select the most effective and efficient combination of implementation techniques, which may entail removing or modifying techniques lacking significant positive effects or showing negative interaction effects. (4) When applicable, consider predetermined constraints of the optimization objective when deciding which implementation techniques to include. This approach ensures alignment with broader objectives and resource limitations, thereby facilitating informed decision-making.