Implementation of the Human Papillomavirus School-entry Requirement in Puerto Rico: Barriers and Facilitators Using the Consolidated Framework for Implementation Research

doi:10.21203/rs.3.rs-409665/v1

Download PDF

Research

Implementation of the Human Papillomavirus School-entry Requirement in Puerto Rico: Barriers and Facilitators Using the Consolidated Framework for Implementation Research

https://doi.org/10.21203/rs.3.rs-409665/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 09 Aug, 2021

Read the published version in Human Vaccines & Immunotherapeutics →

Version 1

posted

You are reading this latest preprint version

Background: In 2018, Puerto Rico (PR) enacted a Human papillomavirus (HPV) vaccine school-entry requirement for students ages 11 to 12. Using the Consolidated Framework for Implementation Research (CFIR), we aimed to identify potential barriers and facilitators of this implementation.

Methods: We conducted a total of 36 qualitative interviews with key informants who were stakeholders from different organizations (Department of Health, Schools, Healthcare Providers, and Community organizations in favor of the requirement) from July 2018 to January 2020. Three researchers performed the interview guide, data coding, and analysis according to the CFIR framework. We evaluated construct rating variability between the organizations to determine barriers and facilitators.

Results: The strongest facilitators determined under the CFIR construct include the stakeholder's awareness of the parent's and student's needs to meet the HPV school-entry requirement. Other facilitators include initiatives for school-entry policies and the relative advantage of this requirement over different strategies. The strongest barriers included the cost for private providers to administer the HPV vaccine, the negative influence of social media about the vaccine, which affected parents' acceptance, and the lack of school nurses as available staff resources for the school entry requirement.

Conclusions: Findings from this study can be used to improve implementation (adaptations/modifications) and inform other US states and countries in earlier stages of consideration of the adoption of similar immunization policies. Most barriers can be modifiable with the implementation of educational programs/training across schools, considering that they are the first line of response to parents of this school entry requirement.

Health Economics & Outcomes Research

CFIR

HPV vaccine

school-entry requirement

Hispanic

This study is one of the few that explored the barriers and facilitators of the HPV school-entry vaccine requirement using the CFIR framework. This valuable tool leads us to evaluate the implementation process from multiple settings and stages.
Results have shown the importance of capacity training to school staff and health stakeholders to boost parents' capacity and engage parents who are unaware or hesitant to vaccinate.
These findings provide new information about challenges that emerged in the implementation. These factors that support or impede the process will help guide other implementers that want to replicate HPV-school entry policies in other jurisdictions or countries.

The Advisory Committee on Immunization Practices (ACIP) recommended the HPV vaccine as part of the routine vaccination for girls (June 2006) and boys (October 2011) [1] age 11–12 years. This initiative was in response to the high incidence of HPV infection and HPV-related cancers affecting about 35,300 of the United States (US) population each year [2]. Based on this recommendation, at least 42 jurisdictions have introduced legislation regarding the HPV vaccine, ranging from providing funding for the HPV vaccine, to educational programs directed to awareness regarding HPV school-entry requirements [3, 4].

School-entry requirements for vaccines have been an effective strategy to increase child immunization rates and decrease the incidence of vaccine-preventable diseases [5, 6]. For the HPV vaccine, five states or territories/jurisdictions in the US (Rhode Island, Virginia, Puerto Rico [PR], the District of Columbia [DC], and more recently Hawaii) have enacted this policy [3]. Opt-out provisions and students’ characteristics vary. For Virginia [7] (enacted in 2008) and Washington DC [3] (2009), this policy requires that all girls entering sixth grade should be vaccinated against HPV. However, parents could opt-outs through medical, religious, or personal beliefs exemptions [8]. In 2015, Rhode Island implemented this policy, gradually, in which the school requirement indicates that seventh graders were required to have the first dose, eighth graders were required to have the second dose, and ninth graders were required to have the third dose [9].

PR has a higher incidence of cervical and penile cancers when compared with various racial/ethnic groups in the US [10, 11]. In addition, both the incidence of cervical cancer and anal cancer are increasing in this population. Specifically for cervical cancer, incidence rates showed an increase of 2.4% per year from 2001 and 2017 in Puerto Rico [12] while for anal cancer, incidence increased 4.90% from 2001 and 2016 [13]. Due to its public health relevance, in 2018 the Secretary of Health of Puerto Rico implemented the HPV vaccine school-entry as a requirement to all students ages 11–12 years old for entering school [14]. Since the implementation enactment, two different amendments to this requirement have occurred: (1) extension of school-entry requirement's age range to 14 years for the 2019–2020 academic school year [15], and (2) extension of the age range to 16 years for 2020–2021 [16]. The last jurisdiction to enact an HPV school entry requirement in the US was Hawaii (July 2020) for 6th graders males and females [17].

The effectiveness of school-entry mandates to reach optimal HPV vaccination uptake might be affected by several factors that can impede its successful implementation. There is plenty of qualitative research conducted in the US that addresses HPV vaccination uptake. These studies have identified that low parental acceptance [18, 19], invasion of parental autonomy (governmental interference) [19], limited providers recommendation [19–21], vaccine insecurity [19], the stigma associated with vaccines against STIs [19], financial burden [3, 18], the flexibility of opt-outs [22], and lack of knowledge [22, 23] are related to low HPV vaccine uptake. Although most stakeholders interviewed prior in the literature were part of the clinical setting (including healthcare providers), few studies have included stakeholders from other organizations such as school and government (Department of Health) [22, 24, 25].

Understanding the process of the implementation of mandatory vaccination for preventable diseases is critical to determine the facilitators and barriers across different settings or contexts that could be involved in its success. The Consolidated Framework for Implementation Research (CFIR), a meta-theoretical framework, has been used across various studies as a practical application to examine the barriers and facilitators across the implementation process [26–28]. This framework has been used previously to describe similar interventions related to HPV vaccination in adolescents [29–31]. CFIR allowed us to obtain a full scan of the HPV school-entry requirement, including barriers and facilitators and the magnitude, strength, and direction of the implementation. The purpose of our study is to evaluate the implementation process of the HPV school-entry requirement in PR using the CFIR framework. To our knowledge, the Human Papillomavirus Policy Implementation Vaccination (HPV-PIVac) is the first study in PR to explore the barriers and facilitators of this HPV vaccine policy from the perspective of stakeholders, concurrently with the implementation of this critical immunization requirement.

Study design and participants

This study used qualitative semi-structured interviews with 46 key informants (KI) who were stakeholders from: The Puerto Rico Department of Health (DOH), school systems (either public or private), healthcare organizations island-wide, as well as community organizations in favor of the school-entry requirement. Additionally, participants against this implementation were part of the total sample interviewed (n = 10). However, this group analysis will be presented apart because their interview analysis mainly focused on themes outside of our selected framework. Participants were selected by recommendation from the Studys’ Community Advisory Board members and from other KIs. Participants had to meet the following inclusion criteria: older than 21 years old, work in an area related to education or health, collaborated with the HPV school-entry requirement in their healthcare setting or organization and agreed to audio-record the interview.

Data collection

Interviews were conducted from July 2018 to January 2020. Only the first interview [date: 07/13/2018] was performed prior to the implementation of this school-entry requirement. Participants were approached either face-to-face in the PR Comprehensive Cancer Center facilities, in their workplace, or by phone. Interviews were audio recording, lasted from 60 to 90 minutes, and were led by one of four investigators: VC, RS, DM, and MR.

Conceptual Framework

The CFIR consists of 39 constructs organized into five major domains that focus on different stages of the implementation [27, 32]. For this study, the CFIR was used to develop the interview guides, data coding and analysis. The guides were similar across the four KIs organizations, although we adapted the questions to each group's expertise area (i.e., health, education). The interview guide had relevant constructs selected under each of the domains. Examples of the five domains and constructs included: 1) Characteristics of individuals (knowledge, attitudes, and beliefs about HPV vaccination), 2) Outer setting (perceived needs to immunization services in the community) 3) Inner setting (Access to knowledge and information about the implementation, staff resources), 4) Intervention characteristics (actual practices, advantages and disadvantages of this implementation), and 5) Process (leadership and compromise by implementers with HPV immunization) among others.

Interview analysis

Data coding

We conducted a series of steps for this qualitative analysis using CFIR as our framework. First, we adapted and developed a new codebook (from the CFIR official website) [32] with the framework constructs and additional constructs that emerged through the analysis (see Additional file 1-codebook supplementary table). The research team discussed and modified the codebook as they coded the first interviews. All the interviews were digitally recorded and transcribed verbatim, using qualitative data Atlas.ti (version 8.4.23.0) software package [33]. We then coded the quotes based on the codebook constructs and definitions. Additional constructs relevant to the implementation were created and included as subthemes of the CFIR original constructs. These additional codes were evaluated independently of the CFIR constructs but within the same theme (e.g., new code created as a sub-construct: unawareness (independent construct) below parent's needs and resources (CFIR construct).

Then, we selected the constructs most cited for this analysis (n = 47). The data and analysis trustworthiness was verified independently and discussed by three investigators (VC, RS, and DM).

Construct ratings

After coding the quotes (n = 1,398), we used a rating system adapted from the CFIR website to qualify the selected constructs [32]. From this analysis, we excluded the constructs of recommendations and job description because they were primarily informative. The rating system was based on two concepts: valence and strength. Valence refers to the factors in the quote that influence positive, negative, or have no influence (neutral) on the school-entry implementation. While strength refers to how strong is the influence of the content of the quote on the implementation, giving values from 0 (not influence), 1(low influence), and 2 (high influence). After rating each quote, we determined the most frequent rating value to the constructs stratified by type of organization, which we denominated as 'rating's score mode' (Table 2).

Analysis and interpretation

After the assignment of ratings per construct, we summarized the results in a rating matrix, considering the pattern of ratings grouped by the organizations interviewed, which we denominated 'variability of the ratings' (Table 2). This matrix led the research group to visually identify patterns in the construct ratings between the four organizations interviewed to determine multilevel barriers and facilitators to the school-entry requirement. We compared the differences in rating scores by row for the variability analysis, considering each organization's construct rating score. The following classification was used: ‘not’ when the scores of the construct ratings were the same for each organization; ‘low’ when the value was mixed but only in strength; and ‘high’ when both strength and valence were different. Those constructs that were categorized to have low or not variability that presented positive ratings were classified as ‘facilitators' or ‘barriers' (if presented negative ratings). Additionally, we consider the sum of the ratings by construct (row) to determine the strongest facilitators and barriers to be shown in the descriptive results. On the other hand, those constructs that had high variability between the organizations were categorized as ‘mixed' (both facilitator and barrier).

Characteristics of the Participants

Table 1 presents the role of each of the KI interviewed. Most were from the school system (n = 15) followed by the Department of Health and healthcare Providers (n = 9 for each group) and community organizations in favor of the school-entry requirement (n = 3). Within the school system, school nurses and directors were most interviewed (n = 5 for each group).

Table 1

Distribution of KI Participants, regarding to stakeholder group and role within the organization (n = 36).
Stakeholder group	Role	n (%)
Department of Health
Program director	Senior administrative supervisor with knowledge or experience in immunization	3 (8.3)
Medical consultant	Professional with experience in advising about immunization	1 (2.8)
Promotion manager	Promotion of vaccination facts, data, and requirements in campaigns and social platforms	1 (2.8)
Program coordinator	In charge of vaccination data quality	2 (5.6)
Program epidemiologist	Assist in the data quality and develop statistics	1 (2.8)
Health educator	In charge of promoting HPV vaccine and other health topics in schools	1 (2.8)
Schools
School directors	Supervisors in charge of private or public schools	5 (13.9)
School nurses	Regional nurses in charge of a series of schools to monitoring vaccine data and vaccine school requirements, among other roles	5 (13.9)
Administrative assistant	Assistant of the school directors in charge of keep record and monitoring the HPV school-entry policy	4 (11.1)
Nurse supervisor	Administrative supervisors in the Central Offices of the Department of Education	1 (2.8)
Healthcare providers
Clinical nurses	Immunization nurses of the Federally Qualified Health Centers (FQHCs)	2 (5.6)
Pediatrician	A physician from private insurance who have the license to vaccinate	1 (2.8)
Gynecologist	Physicians and investigators specialized in HPV	2 (5.6)
Dentist	Academic and investigator in oropharyngeal cancer and HPV related cancers	1 (2.8)
Health Educators	Health providers from the FQHCS	2 (5.6)
Vaccination Program director	Physician in charge of the vaccination clinic of a private Hospital	1 (2.8)
Organizations in favor of the policy
Advocacy organizations	Representatives from two organizations in PR in favor of the school-entry policy who are committed to advocating for the immunization of the entire population through education, to prevent future diseases.	2 (5.6)
Religious	Priest from the Catholic denomination who bring their opinion about the HPV school-entry policy in PR.	1 (2.6)
Total		36 (100)

Distribution of CFIR construct ratings by organization

We identified a total of 32 constructs aligned with CFIR constructs (out of 39 original CFIR constructs) and created an additional 15 constructs for a total of 47 constructs evaluated for valence and strength (Table 2).

Table 2

Ratings assigned to CFIR construct by organization.
Construct	n quotes	Rating's score mode by organization¹				Total rating score²	Variability³	Category⁴
Construct	n quotes	DOH	Schools	Healthcare Providers	CBOs	Total rating score²	Variability³	Category⁴
I. Intervention characteristics
Implementation source (internal)	2	2	-	-	-	2	Not	Facilitator
Evidence Strength & quality	56	1	1	1	1	4	Not	Facilitator
Adverse effects*	7	-1	-2	0	-	-3	Low	Barrier
Relative Advantage	36	1	2	2	2	7	Low	Facilitator
Adaptability	15	1	-	0	2	3	Low	Facilitator
Trialability	3	-1	-1	-	-	-2	Not	Barrier
Complexity	40	-2	-2	-2	-	-6	Not	Barrier
Vaccine complexity completion*	22	0	-1	-1	-1	-3	Low	Barrier
Design quality and Packaging	19	-2	-2	-2	0	-6	Low	Barrier
Cost	19	-2	-2	-1	-2	-7	Low	Barrier
II. Outer setting
Parent needs & resources	52	2	2	2	2	8	Not	Facilitator
Unawareness*	21	1	2	1	1	5	Low	Facilitator
Parental/patient autonomy*	10	0	1	0	0	1	Low	Facilitator
HPV hesitancy*	43	1	1	0	0	2	Low	Facilitator
Vaccine distrust*	24	2	1	1	0	4	Low	Facilitator
Cosmopolitanism	43	1	0	-2	2	1	High	Mixed
External Policy & incentives	40	2	2	1	2	7	Low	Facilitator
Negative influence of social media	23	-2	-2	-2	-1	-7	Low	Barrier
Excessive government interference	6	-1	-	1	-1	-1	High	Mixed
III. Inner Setting
Network & communications	37	1	2	1	-	4	Low	Facilitator
Culture	4	-2	-	-	1	-1	High	Mixed
Implementation climate	14	2	2	1	-	5	Low	Facilitator
Tension for change	14	-1	2	2	-	3	High	Mixed
Relative priority	37	1	2	1	2	6	Low	Facilitator
Organizational incentives & rewards	4	2	-	-2	-	0	High	Mixed
Goals and feedback	15	1	-	-1	2	2	High	Mixed
Learning climate	1	-	2	-	-	2	Not	Facilitator
Readiness for implementation	72	2	1	0	2	5	Low	Facilitator
DOH not enough	22	-1	-1	-1	-1	-4	Not	Barrier
Leadership engagement	27	2	1	2	1	6	Low	Facilitator
Available resources	54	2	-1	1	-2	0	High	Mixed
Available resources staff*	61	-2	-1	-2	-2	-7	Low	Barrier
Access to knowledge and information	75	-1	-1	-1	-1	-4	Not	Barrier
IV. Characteristics of individuals
Knowledge & beliefs about the innovation	62	2	1	2	1	6	Low	Facilitator
Views & knowledge about exemptions	93	-1	0	-2	-1	-4	Low	Barrier
V. Process
Planning	30	2	-2	2	2	4	High	Mixed
Engaging	4	2	-1	-	2	3	High	Mixed
Formally appointed internal implementation leaders	4	-	2	1	1	4	Low	Facilitator
Champions	10	1	1	2	2	6	Low	Facilitator
External change agents	33	1	1	1	1	4	Not	Facilitator
External change agents-capacity*	14	2	1	1	2	6	Low	Facilitator
External change agents-materials*	13	1	2	1	2	6	Low	Facilitator
Key stakeholders	37	1	1	1	1	4	Not	Facilitator
Implementation participants	41	1	2	2	1	6	Low	Facilitator
Communication to community*	63	2	1	1	1	5	Low	Facilitator
Implementation differences between schools	30	-1	-2	-1	-1	-5	Low	Barrier
Reflecting and evaluating	46	0	0	0	-1	-1	Low	Barrier

1 Score mode: after assigning a rating for each quote, we determined the most frequent rating for each construct stratified by organizations

2 Total rating score: the sum of rating's score modes by constructs

3 Variability: the evaluation of the pattern of ratings by each construct (high, low, not)

4 Category: Classification of facilitator, barrier, or mixed based on the pattern of the construct's rating scores

*Additional new codes created as sub-constructs of the CFIR original framework. These new codes were coded and evaluated independently (e.g., mutually exclusive quotes assigned to each of the quotes indicated in this table).

Ratings and variability. Nine (n = 9) constructs have high variability between the organizations interviewed, which implies that organizations have different views and influences about the HPV school-entry requirement. These constructs were: communication between organizations, government interference, goals and feedback, culture, tension for change, organizational incentives and rewards, goals and feedback, available resources, planning, and engaging. Most constructs (38 out of 47) have low (n = 28) or not (n = 10) variability between the construct ratings across the organizations, which implies homogeneity of opinions regarding HPV school-entry requirement. From these, a total of 25 constructs positively influenced the implementation and were classified as facilitators. Most of the facilitators identified were on Process (n = 8) and Outer Setting (n = 6) domains. On the other hand, thirteen (n = 13) constructs negatively influenced implementation in all the organizations and were classified as barriers. Of these barriers, six (n = 6) were on the domains of Intervention characteristics, and three (n = 3) were on the Inner Setting domain.

Details of the findings are described below, categorized by facilitators and barriers to the HPV school-entry policy.

FACILITATORS

The strongest facilitators reflected across all the organizations interviewed (DOH, healthcare providers, schools, and organizations in favor of the implementation) were identified on the domains of Outer Setting and Intervention characteristics. Implementers were aware of the parents' needs and resources to meet with the school-entry policy. Additionally, initiatives for school-entry policies were mentioned as a facilitator for external policies and incentives construct. On the other hand, the fact that this implementation is required gives a relative advantage over different strategies. VOCES (Coalition of Vaccination of Puerto Rico) was the external change agent most mentioned who positively influenced providing education to stakeholders. School and clinical nurses documented messages for engaging parents and adolescents (implementation participants).

Outer Setting

The facilitator with the highest score was parent's needs and resources construct, part of the Outer Setting domain. Implementers demonstrated their awareness and knowledge of parent's needs and resources to accept and understand the school-entry policy. Among those needs and resources that impede a successful vaccination are the HPV vaccine hesitancy, parents' unawareness, vaccine distrust, and the violation of parental/patient autonomy.

Parent's Needs & Resources (total rating score = 8) was the construct with a higher score under the Outer Setting. The implementers demonstrated their awareness and knowledge of the factors that impede that parent vaccinated their youths.

"They [parents] know that they have to vaccinate them [adolescents] with that vaccine specifically for papilloma, but they do not know specifically what the vaccine is [for]. They have heard that [the vaccine] "it is bad, that it is bad". They [parents] know they have to put it [vaccine], but they are not clear about what it is; that is why they are hesitant because what they hear of the vaccine is negative.”

-School director

External Policy and Incentives (total rating score = 7). Initiatives from organizations to spread the HPV school entry requirement were documented, such as campaigns, meetings, development, and support of policies and programs.

“Well, now the Department [of Education] has a ‘family engagement' program, a program to create centers for parents in most schools. This [program] is an area of opportunity to be then able to give orientations to these leading parents in those centers so that they can help us promote vaccines.”

–School nurses’ supervisor

Intervention characteristics

Relative advantage (total rating score = 7). The most common benefit of this implementation is that it is required, increasing the probability that every adolescent of 11–12 years be vaccinated and, hence, increase HPV vaccination rates compared to other strategies.

"I think it would be very successful [the school-entry requirement] because it is the only way we get parents to take their children to vaccinate."

– Health provider

Process

External change agents (total rating score = 6) 62% of quotes mentioned VOCES as an external organization that helped provide education to the community, gave training to school staff, and provide materials with information to assist the Department of Health. Another organization highly mentioned was the pharmaceutical Merck, which aids in providing educational material in schools and clinics.

"An external entity helps to break the existing barrier, and I can attest to that, that this has been the role of VOCES has done so this [HPV school-entry requirement] be achieved."

-Organization in favor of the school-entry policy

Implementation participants (total rating score = 6) under this construct, school and clinical nurses expressed how they engaged parents to vaccinate their youths. Strategies included clear enforcement messages to educate the community about the benefits of vaccination and the use of personal anecdotes to convince them.

[I say to parents] "My children are all protected, and I do not have the distress that these infections [HPV] infect them because I protected them. And many parents that I already have friendship see that I tell them "look at my sons, they have it." There they [parents] say "no, she is not going to expose her children to put something that she does not believe in" because she is a health worker. There they [parents] change their minds.”

–School nurse

Champions (total rating score = 6). Another facilitator was the engagement and dedication from implementers with the school-entry requirement in PR. Most of the interviewees mentioned the Secretary of Health and school nurses as champions committed to this implementation.

"Since he [the Secretary of Health] entered, in 2017, he committed that was going to maintain… that vaccination was important, that he was going to maintain that, and that was the compromise."

–Department of Health vaccination program

Inner Setting

Relative priority (total rating score = 6). This construct documented the importance and the focus of having this requirement in the adolescents' population to prevent future malignancies.

"Super necessary for me [the school-entry requirement], there you have the vast majority of the population, of the target population that you want, where you can control this epidemic at an epidemiological level. So, for me, it's perfect."

-Health provider

Characteristics of Individuals

Knowledge and Beliefs about the implementation (total rating score = 6) showed the implementers' beliefs, and mostly it was optimistic about the vaccine and the requirement.

“I believe in the vaccine, in the benefits of the vaccine to protect young people ... it is a vaccine that certainly the results are not going to see now, but in the long term, [I hope so], that we can see the reduction in the rates of cervical cancer and other types of cancers that are associated."

-Department of Health educator

Other constructs

Other constructs that had a total rating score of 5 points were: unawareness, implementation climate, readiness for implementation, and community communication. Regarding Outer Setting domain, key informants documented the misinformation from parents about the HPV school-entry requirement or the HPV vaccine (unawareness). Under the Inner Setting domain, the construct of implementation climate includes the receptivity of the key informants regarding the HPV school entry requirement, and the construct of readiness of implementation included the organizational commitment to implement the requirement. Additionally, under the Process domain, the construct of communication to the community documented information delivered to the community related to the HPV vaccine or its implementation in schools.

BARRIERS

The strongest barriers to the school entry requirement in PR most mentioned by organizations were in the domains of Implementation Characteristics, Inner and Outer Setting. The cost for private providers to administer the vaccine, the negative influence of social media about HPV vaccine affected the acceptance of parents, and the lack of school nurses as available staff resources for the school entry requirement were the most substantial barriers mentioned that affected the implementation.

Implementation characteristics

Barriers mentioned in the Implementation Characteristics domain reflect issues with the cost of the vaccines to private providers, the program's complexity, and the content of the material presented to the community.

Cost (total rating score= -7). The main barrier was the cost of the vaccine for private providers. Participants mentioned barriers related to the vaccine coverage by the health insurance and documented that it is below the vaccine's cost.

"When a [private] physician is going to administer vaccines, he has to buy the vaccine and have it in the office in a cold chain... I have to pay for the storage, to the nurse also I have to pay. The time of the person who enters that vaccine to the PRIR [registry]. The expense of administering the vaccine that the health insurances, if they pay, pays a pittance. In Puerto Rico, I understand that they pay between four and six dollars. In the United States, they pay $ 25.”

–Pediatrician

Complexity (total rating score= -7). One of the difficulties expressed by key informants was guiding and disseminating the school-entry requirement to the community (health providers, school, parents). Participants also mentioned the process of accepting religious exemptions by schools (in instances, religious exemptions are only accepted if 'include what the law says’). Many implementers expressed that the rules to determine if the exemption is valid are not clear and have difficulties dealing with it.

Another theme mentioned by the participants was the delayed implementation process due to Hurricane Maria's arrival on September 20, 2017: The DOH, health providers, and the vaccination clinics suffered disruption and vaccination losses island-wide [34].

"We have had some challenging years because we had many stumbling. The hurricane has left us ... Not only us here, the Department of Health, the entire island: the doctors, the vaccination centers, VFC, all of those suffered a lot of damage."

- Department of Health

Design quality and packaging (total rating score= -6). The promotion used to inform about the vaccine initially affected parents and the community's acceptance because they focused on it being a vaccine that prevents a sexually transmitted infection.

"How they have marketed it [HPV vaccine] in the beginning saying and stimulating that it is a vaccine that prevents a sexually transmitted infection is one of the things that, perhaps most affected the acceptance of the vaccine worldwide."

–Department of Health vaccination program

Inner Setting

Barriers mentioned in the Inner Setting reflected the need for training to implementers and the lack of materials and human resources to attend the school entry mandate's demands.

Available staff resources (total rating score=-7) Key informants discussed the need for more school nurses' staff to implement this requirement. In many schools, the administrative assistant or the principal must assume this duty, representing an additional responsibility and work overload. Regional nurses of public schools also take this role. Participants discuss the excess of work this task means, as one nurse can have up to 20 schools assigned, and the time to dedicate to each school is limited.

"The difficulty we have is in the limited amount of [school] nurse resources that we have. Now we are blessed this year because we have. But if we don't have this resource, it limits us because the roster of my nurses, we only have 33."

–School nurses' supervisor

Outer Setting

Negative influence of social media (total rating score= -7). Key informants expressed how the misleading information about the HPV vaccine posted on social media affected the acceptance of HPV by parents.

"Regarding HPV vaccine, I think they [parents] have received more negative information through the social media than information by professionals who have adequately guided them for this vaccine, which is why many parents are hesitant to this vaccine."

–School nurse

Other constructs

Related to the Process domain, the construct of implementation differences between public and private schools had a total score of -5 points. This construct documented the organizational differences in engaging the same school entry policy between public and private schools.

Two years after implementing the HPV school entry policy in PR, this study described the factors that facilitate or impede the successful implementation of this policy in this population and the magnitude of the influence of those factors on the implementation. Our study shows high variability across organizations interviewed concerning nine constructs related to Process, Outer Setting and Inner Setting. On the other hand, our study's main findings revealed 25 CFIR constructs that showed low or not variability between the organizations interviewed. These constructs facilitated or represented a challenge to the school-entry requirement implementation. One of the most influential facilitators was constructs of the Outer Setting domain, as the participants acknowledge the parents' needs in understanding the policy. Other facilitators include their strengths in using personal experience and testimony to promote the vaccine and the policy (implementation participants) and the influence of champions (community coalitions) that provide health providers training and capacity. Participants’ beliefs about the HPV vaccine and relative advantage of having this requirement were part of the highlights that facilitated the implementation. On the other hand, the cost of the HPV vaccine and the physicians' low reimbursement for services related to the vaccine administration, storage, and operational costs were among the primary barriers mentioned by key informants. Moreover, some of the strongest barriers documented include the complexity in informing the school-entry requirement to the community, the lack of available school nurses, and the negative influence of social media about the HPV vaccines’ promotion.

Despite the consensus in most of the constructs across organizations, nine constructs reflected high variability in opinions about the implementation between the organizations interviewed. These results demonstrate two things: first, healthcare provider organization participants’ had different views when compared with all other organizations interviewed. For example, the construct of cosmopolitanism, which refers to the degree of networking between organizations, had positive ratings for the Department of Health and community-based organizations. On the other hand, it obtained the lowest rating score (-2) for the healthcare provider organization. Participants from healthcare organizations also had different rating valence regarding the construct of government interference of mandatory vaccination against parents' will compared with the organizations from the Department of Health and CBOs, who had negative ratings. Moreover, in the constructs of organizational incentives & rewards, and goals & feedback, the healthcare providers had negative rating scores compared with the other organizations' positive scores.

On the other hand, participants from school organization had negative ratings compared with all other organizations interviewed in the constructs of planning (the degree to which tasks for implementing the school-entry requirement are developed) and engaging (involving appropriate individuals in the implementation and use of the school-entry requirement through combined strategies). We speculate that these findings of differences in opinions from healthcare providers and school organizations might be due to the lack of information delivered, particularly in the context of the policy's time (August 2018). This event was 11 months after Hurricane Marias' arrival, which triggers other public health problems, leaving aside the priority of the HPV vaccine, which could affect the dissemination of information [34–36]. These interruptions caused doubts in the school personnel particularly and might affect the Department of Health's communication with providers. Although training physicians, journalists, and school staff were conducted prior to implementation, divergences are still observed; hence, more efforts are still needed, as healthcare providers (clinical nurses and physicians) [37] and school nurses [38] are the first lines of contact with parents regarding HPV vaccination.

The construct classified as the strongest facilitator was parents’ needs and resources, which describes the extent to which vaccine recipients' needs are accurately known and prioritized and the barriers to reaching them. Participants demonstrated their knowledge and awareness about the factors that promote vaccine hesitancy in parents. Some of the factors documented by key informants interviewed and mentioned in previous articles by Hispanic and African American parent-level focus groups included: unawareness [39] and misinformation about the benefits [40] of the HPV vaccine, stigma attributed to an STI-vaccine related [30, 39], vaccine distrust due to long-term effects [39–41], effectiveness [41], among others. Additionally, this facilitator is consistent with what has been found in a previous study using CFIR [29]; however, the reasons identified were not the same. As many of these factors have been recognized in focus groups of Hispanic parents, stakeholders must understand the gaps that impede parents' vaccinating their sons and daughters. Especially in those in the first line of contact (school and clinical nurses, pediatricians) documented that they know their population, know their barriers to up-to-date vaccination. These qualities make them a strong source of influence to parents [40, 42].

Three barriers obtained the same highest total rating score. These were available staff resources, negative influence of social media, and cost. Limited staff resources was a barrier mentioned by key informants to discuss mainly the need of school nurses' staff to implement this requirement. Previous studies documented the lack of staff resources allocated to the school-entry vaccination initiatives, leading to work overload by nurses [38, 42]. Other studies using CFIR identified time as a barrier to staff resources that affected productivity [30]. In Puerto Rico, the economic crisis, which was documented to started in 2014, could affect hiring government personnel in the Department of Health and Education Department prior, which can aid, provide support, and monitor the correct implementation of this school entry policy August 2018. The second barrier of social media’s negative influence refers to posts from social media platforms attacking the HPV or the school-entry requirement. Participants were concerned about the information available through the internet (unreliable sources), which are often the most common source accessible and influential to this population of young parents. These findings were consistent with other studies using CFIR expressing frustration about messages of opposition from religious groups because linked sexuality with young girls [31], the sources of information and influences such as the internet [40].

Additionally, the barrier of the cost was mentioned mainly by private physicians (from the healthcare organizations) and focused on the cost associated with the coverage of the vaccine by the health insurances and all the costs associated with the maintenance and administration of the vaccine. Another factor mentioned was the high cost of the vaccine for those who were uninsured, although in Puerto Rico the health insurances are required to cover this vaccine and almost all are eligible for vaccine for children program. The cost of the vaccine as a barrier to patients was also mentioned in previous studies using CFIR [31, 40]. Other studies also mentioned the barrier of the cost associated with the infrastructure, storage, and administration [22, 30].

Strenght and limitations

This study is the first to evaluate the implementation of the HPV vaccine school-entry requirement in Puerto Rico with interviews with stakeholders from different organizations. We found the CFIR framework a useful and practical tool to construct the interview guide and analyzing the data. Because it allows combining qualitative (data coding) and quantitative (construct rating score) to better measure the implementation and make conclusions, we considered this a notable strength. Moreover, all the domains were evaluated in this analysis. We interviewed four organizations that include implementers from multiple sectors involved directly in the HPV vaccine school entry requirement; this helped us see a different and broad perspective of the implementation. This analysis provides new information about challenges that emerged in the implementation and factors that support the process.

As we interviewed four organizations related to the implementation, we acknowledge that there were an organization who could not be included in the study population even though we considered them essential to interview, such as policymakers from the government of Puerto Rico (both in the senate and the House of Representatives), because they were part of the implementation process. Still, the sample of participants already exceeded the sample proposed. Additionally, the number of key informants is unbalanced, as the school organizations have a higher number of participants recruited (n = 15) compared with the lowest number of community organizations (n = 3), which might influence the number of quotes explored and analyzed.

Our study has shown both positive and negative factors that affected the implementation of the HPV school entry requirement in PR. Because school entry requirements are concentrated primarily in school settings, it is crucial to maintain this personnel trained and accessible to this information, as they are the first line of contact with parents [43]. This study highlights the importance of giving training and tools for school and health stakeholders to boost parents’ capacity to engage parents who are unaware or hesitant to vaccinate. On the other hand, health insurance should be regulated to compensate justly to those private physicians with low earnings due to the vaccination process. Additionally, those negative influences of social media should be offset with an increment of social media information from reliable and official sources. Most barriers presented in this study can be modifiable with the implementation of educational programs/training across organizations, considering the first line of response to parents of this school entry policy. These findings generate recommendations for our stakeholders to influence policy implementation improvements and can be used to guide other implementers that want to replicate this implementation in other jurisdictions or countries.

HPV- Human Papillomavirus

PR- Puerto Rico

CFIR- Consolidated Framework for Implementation Research

ACIP- Advisory Committee on Immunization Practices

US- United States

DC- District of Columbia

HPV-PIVac- Human Papillomavirus Policy Implementation Vaccination study

KI- Key informants

DOH- Department of Health

CBOs- Community based organizations

VOCES- Puerto Rico Coalition of Vaccination

Ethics approval and consent to participate:

Ethical approval for the Implementation of School Entry Policies for Human Papillomavirus Vaccination (HPV-PIVac) was secured from the University of Puerto Rico Medical Sciences Campus- Institutional Review Board (Protocol #A8060218)

Consent for publication:

Not applicable- This manuscript does not contain individual person’s data

Availability of data and materials:

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Competing interests:

VCL has received compensation from Merck and Co., Inc. for consultancy in June 2020. APO reported has received personal fees from serving as a consultant for Merck (October 2019) outside the submitted work. The other authors have declared that they have no competing interest.

Funding:

This work was supported by National Cancer Institute (NCI), grant R01CA232743 (“Implementation of School-Entry Policies for Human Papillomavirus Vaccination”) within University of Puerto Rico Comprehensive Cancer Center. The National Cancer Institute had no role in the study design; collection, analysis, and interpretation of the data; writing of the manuscript; or decision to publish. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NCI.

Author’s contribution:

VCL and RSA were involved in the study's design, conceptualization, writing-original draft preparation, investigation, and validation. DMTL and ODM were involved in the formal analysis, writing- reviewing and editing. APO, ELSP, and PCH contributes with writing-reviewing and editing. PCH contributes to the study’s design, formal analysis, investigation, writing- reviewing, and editing. All authors read and approved the final manuscript.

Acknowledgments:

This publication was supported by the Comprehensive Cancer Center of the UPR (a public corporation of the Government of Puerto Rico created in virtue of Law 230 of August 26, 2004 as amended). The content is entirely the responsibility of the authors and does not necessarily represent the official views of the Comprehensive Cancer Center UPR.

We want to thank to Dr. María Fernández, Manuel E. Rivera-Encarnación, Vilnery Rivera-Figueroa; Glizette O. Arroyo-Morales, Luis O. León Martinez, Michelle Ramirez-López, Kevin J. Colón-Rivera, and Andrea N. Suárez-Delgado to their support and collaboration.

Frieden TR, Harold Jaffe DW, Kent CK, Leahy MA, Martinroe JC, Spriggs SR, et al. Human Papillomavirus Vaccination Recommendations of the Advisory Committee on Immunization Practices (ACIP). 2014. http://www.cdc.gov/mmwr/cme/conted.html. Accessed Jun 1 2020
Centers for Disease Control and Prevention (CDC). How Many Cancers Are Linked with HPV Each Year?. 2020. https://www.cdc.gov/cancer/hpv/statistics/cases.htm. Accessed Feb 24 2021
National Conference of States Legislatures. HPV Vaccine: State Legislation and Statutes. 2020. https://www.ncsl.org/research/health/hpv-vaccine-state-legislation-and-statutes.aspx#. Accessed Mar 4 2020.
HPV Vaccine Mandates for Elementary and Secondary Schools. https://www.immunize.org/laws/hpv.asp. Accessed Mar 4 2020
Barraza L, Weidenaar K, Campos-Outcalt D, Yang YT. Human papillomavirus and mandatory immunization laws: What can we learn from early mandates? Public Health Reports. Association of Schools of Public Health; 2016;131:728–31.
The Guide to Community Preventive Services (The Community Guide). What Works Fact Sheet: Increasing Appropriate Vaccination. 2017. https://www.thecommunityguide.org/resources/what-works-increasing-appropriate-vaccination. Accessed Mar 30 2021
Virginia Department of Health. School Requirements–Immunization. http://www.vdh.virginia.gov/immunization/requirements/. Accessed Mar 11 2020
District of Columbia Department of Health. District of Columbia Immunization Requirements School Year 2018-2019. http://dcps.dc.gov/service/medication-and-treatment-school. Accessed Mar 11 2020
National HPV Vaccination Roundtable: Rhode Island HPV Immunization Case Study. Cancer Control Tap. https://smhs.gwu.edu/cancercontroltap/resources/national-hpv-vaccination-roundtable-rhode-island-hpv-immunization-case-study. Accessed Mar 30 2021
Colón-López V, Ortiz AP, Soto-Salgado M, Torres-Cintrón M, Pettaway CA, Puras-Báez A, et al. Penile cancer disparities in Puerto Rican men as compared to the United States population. Int Braz J Urol. 2012;38:728–38.
Ortiz AP, Soto-Salgado M, Calo WA, Hull P, Fernández ME, Colon-López V, et al. Elimination of cervical cancer in U.S. Hispanic populations: Puerto Rico as a case study. Prev Med (Baltim). Prev Med; 2021;144:106336. https://linkinghub.elsevier.com/retrieve/pii/S0091743520303674. Accessed Mar 30 2021
Ortiz AP, Ortiz-Ortiz KJ, Colón-López V, Tortolero-Luna G, Torres-Cintrón CR, Wu C-F, et al. Incidence of Cervical Cancer in Puerto Rico, 2001-2017. JAMA Oncol. American Medical Association (AMA); 2021; doi:10.1001/jamaoncol.2020.7488
Ortiz-Ortiz KJ, Ramos-Cartagena JM, Deshmukh AA, Torres-Cintrón CR, Colón-López V, Ortiz AP. Squamous Cell Carcinoma of the Anus Incidence, Mortality, and Survival Among the General Population and Persons Living With HIV in Puerto Rico, 2000-2016. JCO Glob Oncol. American Society of Clinical Oncology (ASCO); 2021;7:133–43. https://pubmed.ncbi.nlm.nih.gov/33493020/. Accessed Mar 30 2021
CB en Español. Requieren vacuna del virus del Papiloma Humano en escuelas de Puerto Rico – CB en Español. 2017. https://cb.pr/requieren-vacuna-del-virus-del-papiloma-humano-en-escuelas-de-puerto-rico/. Accessed Jun 5 2020.
El Vocero de Puerto Rico. Informan nuevos requisitos de vacunación para próximo año escolar. 2019. https://www.elvocero.com/actualidad/informan-nuevos-requisitos-de-vacunaci-n-para-pr-ximo-a-o-escolar/article_296bf6f8-8263-11e9-971e-4fda85497cfe.html. Accessed Jun 2 2020
Departamento de Salud de Puerto Rico. Requisitos de Vacunación para el Curso Escolar 2020-2021. 2020. http://www.salud.gov.pr/Documents/Requisitos de Vacunas para el Curso Escolar 2020-2021.pdf. Accessed Mar 30 2021
Disease Outbreak Control Division. 2020-2021 School Health Requirements. https://health.hawaii.gov/docd/vaccines-immunizations/school-health-requirements/sy-20-21/. Accessed Mar 11 2020
North AL, Niccolai LM. Human Papillomavirus Vaccination Requirements in US Schools: Recommendations for Moving Forward. Public Health. 2016. 106:1765–70. doi: 10.2105/AJPH.2016.303286. Accessed Apr 24 2020
Vercruysse J, Chigurupati NL, Fung L, Apte G, Pierre-Joseph N, Perkins RB. Parents’ and providers’ attitudes toward school-located provision and school-entry requirements for HPV vaccines. Hum Vaccines Immunother. Taylor and Francis Inc.; 2016;12:1606–14.
Califano S, Calo WA, Weinberger M, Gilkey MB, Brewer NT. Physician support of HPV vaccination school-entry requirements. Hum Vaccines Immunother. Taylor and Francis Inc.; 2016;12:1626–32.
Dorell C, Yankey D, Kennedy A, Stokley S. Factors that influence parental vaccination decisions for adolescents, 13 to 17 years old: National Immunization survey-teen, 2010. Clin. Pediatr. (Phila). 2013. p. 162–70. http://www.ncbi.nlm.nih.gov/pubmed/23221308. Accessed Apr 28 2020
Carhart MY, Schminkey DL, Mitchell EM, Keim-Malpass J. Barriers and Facilitators to Improving Virginia’s HPV Vaccination Rate: A Stakeholder Analysis With Implications for Pediatric Nurses. J Pediatr Nurs. W.B. Saunders; 2018;42:1–8.
Holman DM, Benard V, Roland KB, Watson M, Liddon N, Stokley S. Barriers to human papillomavirus vaccination among us adolescents a systematic review of the literature. JAMA Pediatr. JAMA Pediatr; 2014. p. 76–82. https://pubmed.ncbi.nlm.nih.gov/24276343/. Accessed Jan 20 2021
Cartmell KB, Young-Pierce J, McGue S, Alberg AJ, Luque JS, Zubizarreta M, et al. Barriers, facilitators, and potential strategies for increasing HPV vaccination: A statewide assessment to inform action. Papillomavirus Res. Elsevier B.V.; 2018 5:21–31. http://www.ncbi.nlm.nih.gov/pubmed/29248818. Accessed Feb 24 2020
Kaul S, Do TQN, Hsu E, Schmeler KM, Montealegre JR, Rodriguez AM. School-based human papillomavirus vaccination program for increasing vaccine uptake in an underserved area in Texas. Papillomavirus Res. Elsevier B.V.; 2019;8.
Means AR, Kemp CG, Gwayi-Chore MC, Gimbel S, Soi C, Sherr K, et al. Evaluating and optimizing the consolidated framework for implementation research (CFIR) for use in low- And middle-income countries: A systematic review. Implement. Sci. BioMed Central Ltd.; 2020. p. 17. http://www.ncbi.nlm.nih.gov/pubmed/32164692. Accessed Apr 23 2020
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implement Sci. BioMed Central; 2009;4:50. http://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-4-50. Accessed Feb 20 2020
Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the Consolidated Framework for Implementation Research. Implement. Sci. BioMed Central Ltd.; 2016. p. 72. http://implementationscience.biomedcentral.com/articles/10.1186/s13012-016-0437-z. Accessed Mar 15 2020
Soi C, Gimbel S, Chilundo B, Muchanga V, Matsinhe L, Sherr K. Human papillomavirus vaccine delivery in Mozambique: Identification of implementation performance drivers using the Consolidated Framework for Implementation Research (CFIR). Implement Sci. BioMed Central Ltd.; 2018;13. https://pubmed.ncbi.nlm.nih.gov/30545391/. Accessed Jan 25 2021
Escoffery C, Riehman K, Watson L, Priess AS, Borne MF, Halpin SN, et al. Facilitators and Barriers to the Implementation of the HPV VACs (Vaccinate Adolescents Against Cancers) Program: A Consolidated Framework for Implementation Research Analysis. Prev Chronic Dis. Centers for Disease Control and Prevention (CDC); 2019;16:180406. http://www.cdc.gov/pcd/issues/2019/18_0406.htm. Accessed Mar 15 2020
Selove R, Foster M, Mack R, Sanderson M, Hull PC. Using an Implementation Research Framework to Identify Potential Facilitators and Barriers of an Intervention to Increase HPV Vaccine Uptake. J Public Health Manag Pract. Lippincott Williams and Wilkins; 2017;23:e1–9. http://www.ncbi.nlm.nih.gov/pubmed/27902559. Accessed Mar 15 2020
The Consolidated Framework for Implementation Research | Technical Assistance for users of the CFIR framework. Available from: https://cfirguide.org/. Accessed Feb 20 2020.
ATLAS.ti: The Qualitative Data Analysis & Research Software. https://atlasti.com/. Accessed Feb 20 2020.
Luna-Pinto S, Rivera A, Cardona I, Rijo C, Alvarez V, Rodriguez J, et al. Restoring Immunization Services Provided by the Vaccines for Children Program in Puerto Rico After Hurricanes Irma and Maria, 2017-2019. J Public Heal Manag Pract. 2020; doi: 10.1097/PHH.0000000000001193
Gay HA, Santiago R, Gil B, Remedios C, Montes PJ, López-Araujo J, et al. Lessons Learned From Hurricane Maria in Puerto Rico: Practical Measures to Mitigate the Impact of a Catastrophic Natural Disaster on Radiation Oncology Patients. Pract Radiat Oncol. Elsevier Inc.; 2019;9:305–21. practicalradonc.org. Accessed Mar 31 2021
Pullen LC. Puerto Rico after Hurricane Maria. Am J Transplant [Internet]. Blackwell Publishing Ltd; 2018;18:283–4. https://onlinelibrary.wiley.com/doi/abs/10.1111/ajt.14647. Accessed Mar 31 2021
Gilkey MB, McRee AL. Provider communication about HPV vaccination: A systematic review. Hum. Vaccines Immunother. Taylor and Francis Inc.; 2016. p. 1454–68. https://pubmed.ncbi.nlm.nih.gov/26838681/. Accessed Mar 8 2021.
Brabin L, Stretch R, Roberts SA, Elton P, Baxter D, McCann R. The school nurse, the school and HPV vaccination: A qualitative study of factors affecting HPV vaccine uptake. Vaccine. 2011;29:3192–6. Available from: http://www.ncbi.nlm.nih.gov/pubmed/21354481. Accessed Apr 2 2020
Fernández ME, Le YCL, Fernández-Espada N, Calo WA, Savas LS, Vélez C, et al. Knowledge, attitudes, and beliefs about human papillomavirus (HPV) vaccination among Puerto Rican mothers and daughters, 2010: A qualitative study. Prev Chronic Dis. Centers for Disease Control and Prevention (CDC); 2014;11. https://pubmed.ncbi.nlm.nih.gov/25474384/. Accessed Mar 8 2021
Liebermann E, Devanter N Van, Frías Gúzman N, Ompad D, Shirazian T, Healton C. Parent-Level Barriers and Facilitators to HPV Vaccine Implementation in Santo Domingo, Dominican Republic. J Community Health. Springer; 2020 45:1061–6. https://pubmed.ncbi.nlm.nih.gov/32394119/. Accessed Feb 25 2021
Galbraith-Gyan K V., Lechuga J, Jenerette CM, Palmer MH, Moore AD, Hamilton JB. African-American parents’ and daughters’ beliefs about HPV infection and the HPV vaccine. Public Health Nurs. Blackwell Publishing Inc.; 2019;36:134–43. https://pubmed.ncbi.nlm.nih.gov/30548324/. Accessed Mar 8 2021
Dubé E, Gagnon D, Clément P, Bettinger JA, Comeau JL, Deeks S, et al. Challenges and opportunities of school-based HPV vaccination in Canada. Hum Vaccines Immunother. Taylor and Francis Inc.; 2019;15:1650–5. http://www.ncbi.nlm.nih.gov/pubmed/30633622. Accessed Apr 2 2020
Washburn T, Devi Wold A, Raymond P, Duggan-Ball S, Marceau K, Beardsworth AM. Current initiatives to protect Rhode Island adolescents through increasing HPV vaccination. Hum. Vaccines Immunother. Taylor and Francis Inc.; 2016. p. 1633–8.

Download PDF

Journal Publication

published 09 Aug, 2021

Read the published version in Human Vaccines & Immunotherapeutics →

Version 1

posted

You are reading this latest preprint version

Implementation of the Human Papillomavirus School-entry Requirement in Puerto Rico: Barriers and Facilitators Using the Consolidated Framework for Implementation Research

Status:

Journal Publication

Version 1

Abstract

Contributions To The Literature

Background

Methods

Study design and participants

Data collection

Conceptual Framework

Interview analysis

Data coding

Construct ratings

Analysis and interpretation

Results

Characteristics of the Participants

Distribution of CFIR construct ratings by organization

FACILITATORS

Outer Setting

Intervention characteristics

Process

Inner Setting

Characteristics of Individuals

Other constructs

BARRIERS

Implementation characteristics

Inner Setting

Outer Setting

Other constructs

Discussion

Strenght and limitations

Conclusion

List Of Abbreviations

Declarations

References

Supplementary Files

Status:

Journal Publication

Version 1