Evaluation of a Large-Scale School Wellness Intervention Through the Consolidated Framework for Implementation Research (CFIR): Implications for Dissemination and Sustainability

doi:10.21203/rs.3.rs-647714/v2

Background

Although comprehensive approaches to student health promotion through wellness policy implementation show potential for scale-up, scant empirical research has been conducted to meaningfully integrate dissemination and implementation (D&I) in evaluation procedures. Such application is necessary to understand how and why interventions are sustained over time. The aims of this study were to investigate: 1) implementation determinants of adoption, fidelity, and penetration for school-wide wellness programming; and 2) nuanced determinants between schools with prior experience and those new to the program, to enhance tailored implementation support and sustainability.

Methods

The School Wellness Integration Targeting Child Health (SWITCH®) capacity-building intervention was adopted in 52 elementary and middle (22 new; 30 experienced) schools across Iowa, United States in the 2019–2020 academic year. The Consolidated Framework for Implementation Research (CFIR) guided development of mixed methods data collection and analysis protocols, adapted to school settings. Determinants from CFIR were linked to three key implementation outcomes: 1) fidelity/compliance to established quality elements; 2) adoption of best practices in multiple settings; and 3) penetration of behavior change practices across classrooms and grade levels. Organizational readiness/capacity was also assessed at baseline. Interview data were scored using a systematic process; each CFIR domain was assigned a score (ranging between -2 and +2) to denote either a positive or negative influence on implementation. Independent t-tests were conducted to capture potential differences between new and experienced schools, followed by Pearson bivariate correlation analyses to determine relationships between CFIR determinants and implementation outcomes.

Results

In experienced schools, fidelity/compliance (t=-1.86 p=.07) and adoption (t=-2.03 p=.04) were slightly higher than new schools. The CFIR determinants of innovation source, culture and relative priority, and leadership engagement were positively related to implementation outcomes, whereas tension for change and networks and communications were observed negative determinants. Unique negative determinants for new schools comprised relative advantage, engaging key stakeholders, and reflecting/evaluating, among others.

Conclusions

Findings provide specific areas of need where implementation support may be most valuable, especially for new schools undertaking a comprehensive program such as SWITCH. Tailoring implementation support to both new and existing settings will ultimately enhance dissemination and sustainability of evidence-based interventions.

Health Economics & Outcomes Research

Health Policy

Internal Medicine

physical activity

nutrition

sedentary behavior

dissemination

children

health promotion

obesity prevention

To date, implementation science models such as the Consolidated Framework for Implementation Research (CFIR) have been used widely in healthcare settings, but with hardly any application to the school setting
This study is the first of its kind to investigate relationships between CFIR implementation determinants and outcomes within a large-scale school wellness dissemination effort
Our findings can provide pragmatic support to school-based researchers wishing to apply implementation science methods to their work and highlight the factors which may be drivers of implementation success. This will ultimately enhance the selection and tailoring of implementation strategies for school-based health promotion interventions

School-based health promotion interventions have been shown to have a positive impact on promoting student physical activity and nutrition behaviors (1–5); however, systematic application of dissemination and implementation science (D&I) frameworks are needed to advance the gap between research and practice (6, 7). Furthermore, despite the promise of comprehensive programs, scant research exists to illustrate steps to sustain programs over time, limiting impact on overall population health (8, 9).

The present paper reports on the capacity-building process employed in a school wellness initiative called SWITCH® (School Wellness Integration Targeting Child Health). With support from a United States Department of Agriculture (USDA) grant, SWITCH has been systematically tested and refined over the past 5 years (10–14). The original Switch program was designed to help students “switch what they do, view, and chew” emphasizing the importance of “switching up” to 60 minutes or more of physical activity, ≥ 5 servings of fruits and vegetables, and < 2 hours of non-educational screen time per day (15). The D&I approach grounding this USDA project facilitated the transition from an evidence-based program (i.e., Switch) to an evidence-based process (i.e., SWITCH) for health promotion in schools.

Previous research has examined: the effectiveness of SWITCH to elicit improvements in children’s health behaviors through randomized and non-randomized trials (12, 15, 16); the cost-effectiveness and feasibility of moving to an online format for training and implementation (14); acceptability and feasibility of educational modules in the classroom, physical education, and lunchroom settings (17–19); validity of school readiness and wellness environment assessment tools (20, 21); and different implementation strategies to enhance overall implementation (10, 11, 13).

Consistent with general D&I recommendations, the ongoing implementation of SWITCH is evaluated through multiple measures to understand what influences implementation processes (https://www.iowaswitch.org/). The Consolidated Framework for Implementation Research (CFIR) (22, 23), referred to as a determinants framework in the D&I literature, has been particularly helpful in understanding factors influencing implementation of SWITCH. Specifically, CFIR comprises 39 constructs housed within six key constructs: Intervention Characteristics (factors within the intervention itself such as cost and complexity); Outer Setting (factors external to the implementation setting such as policy); Inner Setting (factors within organization such as networks, culture); Readiness for Implementation (organizational and individual capacity for implementation); Characteristics of Individuals (implementation leaders’ confidence and motivations to implement); and Implementation Process (practices that facilitate implementation such as planning and executing). Such framework has been used predominantly in healthcare settings to investigate determinants of implementation (24–27), with growing application to school and community settings (28, 29). The CFIR website (www.CFIRguide.org) provides comprehensive resources for researchers conducting qualitative and mixed methods evaluation to ground their analysis through systematic coding of interview/qualitative data to facilitate interpretation (26). To date, however, no application of these systematic coding methods has been documented for school and/or community-based implementation evaluations, warranting adaptation to enhance use in non-clinical settings. In a recent evaluation of SWITCH, interview data analyses revealed that schools who had at least one year of experience with the program reported that they were able to better navigate SWITCH and its core components (11). However, we did not fully integrate the interview and implementation fidelity data through use of CFIR coding procedures, and this hindered our ability to understand determinants that linked to specific implementation outcomes.

To date, few studies have directly examined the relationships between implementation determinants and outcomes in school-based health promotion research (30, 31). Further, recent research highlights that readiness for implementation is the most prevalent barrier to sustaining evidence-based interventions in schools (9), warranting greater emphasis on organizational readiness/capacity-building frameworks. Thus, the primary purpose of the study was to systematically investigate factors that influence implementation outcomes through an integrated mixed methods analysis, using D&I methods previously developed for healthcare settings and adapting them to address school-based implementation. Second, we sought to explore relationships among implementation profiles and examine nuanced factors which influence implementation in new versus experienced schools. Results of this study will provide critical information which may help inform implementation strategies for scale-up and sustainability.

A mixed methods implementation study grounded in the CFIR was conducted to evaluate the relationships between implementation determinants and outcomes in the 2019-2020 cycle of SWITCH. Evaluation approaches followed recommended data collection and analytic methodologies of CFIR, developed by Damschroder and colleagues (22, 26). To our knowledge, this is the first documented adaptation of the CFIR mixed methods protocols with the goal of understanding relationships between implementation determinants and outcomes within a school health promotion context.

Participants and procedures

In the 2019-2020 iteration of SWITCH, 52 schools were enrolled, 30 (57%) of whom had already taken part in the program in previous years. Demographic information for these schools is shown in Table 1. The cyclical training (fall) and implementation (spring) process of SWITCH across the academic year facilitates a continuous quality improvement process (32), whereby feedback from schools and implementation outcome data drive modifications to the program each year. More information about the training process can be found in Additional File 1, our previously published article (11), and the program website (https://www.iowaswitch.org/). Briefly, schools were asked to form a wellness team which comprised three members of staff across different school settings (e.g., classroom teachers, physical education, food service, other teachers, administration, counselors, nurses, etc.) and to register prior to the beginning of the academic year. Following registration, schools were asked to attend a total of four webinars and an in-person conference during the fall semester, as well as complete several pre-program audit tools. The implementation phase spanned a 12-week period from January–April of 2020, but due to the coronavirus (COVID-19) outbreak, schools were forced to close in Iowa on March 13^th thus forcing a transition to virtual communications/implementation after week 8 of the program. It was not possible to capture final outcome data, but schools completed the midpoint evaluation of school implementation. Below we outline data sources for implementation outcomes and determinants, and the steps taken to rigorously analyze these data.

Measurement of Implementation Outcomes: Adoption, Fidelity, and Penetration

The field of D&I offers many frameworks and theories to help researchers and practitioners discern why evidence-based practices are or are not implemented in routine care. Regarding implementation outcomes frameworks, the framework by Proctor and colleagues (33) conceptualized several distinctive outcomes that are important to include within implementation evaluations: 1) acceptability (the degree to which an innovation is a perceived good fit), 2) adoption (intent to implement), 3) appropriateness (degree of compatibility within setting), 4) cost (to implement, value for money), 5) feasibility (possibility of successful implementation), 6) fidelity/compliance (executed as intended), 7) penetration (reach within setting), and 8) sustainability (long-term impact). For the purpose of this study, we chose to examine the determinants of adoption, fidelity, and penetration among schools enrolled in SWITCH due to the heavily integrated implementation practices needed to create systems change in the school setting.

Adoption is operationalized by Proctor and colleagues(33) as “intention, initial decision, or action to try or employ an innovation or evidence-based practice” (p. 69). Thus, we measured adoption through implementation surveys at the six-week mark, examining uptake of best practices in various settings (use of curricular modules, posters, reinforced themes through discussion and tracking). Each best practice was scored as 0 (not at all implemented), 2 (somewhat implemented), and 3 (fully implemented) and a summed score was generated based on the average of each component, to give possible range of 0–9.

Fidelity relates to “the degree to which an intervention was implemented as it was prescribed in the original protocol or as it was intended by the program developers” (p. 69) (33). The quality elements of SWITCH comprise, wellness team meeting (ideally at least once per week), using SWITCH website to promote student behavior tracking, engaging parents and other stakeholders, and integration of SWITCH modules/posters across the school setting. Fidelity therefore was calculated by using a summed score of quality elements which were scored the same way as best practices, giving a possible range of 0–12.

Finally, penetration is defined as the “integration of a practice within a service setting and its subsystems.” (p.70) (33) this was calculated by determining the number of participants who used or interacted with an evidence-based practice, divided by the total number of participants eligible or within the sample. Since the behavioral tracking and goal setting interface is an integral component for students (12), it provides a good indicator of how many students are actively engaged in SWITCH within each school, thus providing data on penetration. We used data from SWITCH behavior tracking across weeks 1–8 (to account for COVID-19-related school closures). These data are presented as a decimal score (range 0–1.0, translated to 0–100%).

Implementation Determinants

Organizational readiness

To assess baseline readiness for implementation, the School Wellness Readiness Assessment (SWRA) tool (21) was previously developed in line with the theory of organizational readiness for change (34, 35) and community capacity-building frameworks (36). The SWRA applies these concepts to school environments, taking into account the unique, complex structure and specific settings within schools that impact student health, including classrooms, physical education, and lunchroom settings, and the broader school leadership and cultural context.

The SWRA includes questions across four subscales designed to assess setting-specific and school-wide wellness readiness: classroom readiness, physical education (PE) readiness, food services readiness, and school readiness. The SWRA items were assessed using a 5-point scale (strongly disagree, disagree, neither agree nor disagree, agree, and strongly agree scale, coded as 0, 1, 2, 3, and 4, respectively). A copy of the SWRA is provided in Additional File 2. Wellness teams completed the 40-item SWRA through the program website. Scores for each of the subscales were calculated by averaging together the item responses in each section with higher scores representing higher states of readiness in specific settings and schools.

Qualitative interviews grounded in CFIR

Following procedures developed by Damschroder and colleagues (26, 27, 37), an interview guide was developed which aimed to understand the influence of each CFIR domain on implementation of SWITCH (see Additional File 3). Questions were open-ended, examples included, “What is your perception of the quality of the modules, posters, and other SWITCH materials that were provided?” (Innovation Characteristics – Design Quality and Packaging) and “How do you think your school culture affected the implementation of SWITCH programming?” (Inner Setting – Implementation Climate). Interviews were conducted by a qualitative and survey methodologist to ensure impartiality in responses from school wellness teams. Participants were encouraged to respond openly and candidly about their experiences with implementation and did not set a time limit on these conversations to ensure in-depth understanding of each context and implementation climate. Interviews lasted between 31 and 63 minutes, conducted through video conferencing software (i.e., Zoom), and transcribed verbatim.

Qualitative data coding and case memos

The structure of the interview guide facilitated a predominantly deductive data analysis approach, in that each of the questions corresponded to a construct within each of the framework domains (26). However, we remained open, such that any themes that emerged through inductive approaches were included in our analyses (38). First, the lead and second author met to develop a coding consensus document (Additional File 4), which described each CFIR construct and anticipated potential responses and themes that would emerge through the data. Applying the CFIR systematic coding approach facilitated the assignment of numerical scoring to the qualitative data, such that if a particular construct was deemed to have a positive influence on implementation based on interview responses, a score of +1 or +2 was assigned for that construct. Conversely, if a construct was deemed to be a negative influence, a score of -1 or -2 was given. If it was not clear whether a positive/negative influence manifested, a score of 0 was given, a score of “X” was used for mixed results (later coded as 0 for quantitative analysis, see Additional File 5 for details on CFIR rating rules) (26).

Second, to establish inter-rater reliability, the two coders selected five transcripts and created independent case memos using the CFIR memo templates (37). Scores were compared and a percent agreement score was calculated, if the overall agreement score was <80%, the coders met to ensure consensus before coding another set of five transcripts. Once ≥80% agreement was met, the second author coded the remaining transcripts, before a randomly selected set of 5 transcripts was reviewed by the lead author. All coding was completed in memo documents (see Additional File 6). Finally, to facilitate content analysis and interpretation of trends in interview data, all memos were entered in to NVivo qualitative analysis software and coded into respective nodes, following the CFIR codebook template (37).

Data Analysis

All school demographic, implementation outcome, and implementation determinant data were merged using SAS software (Version 9.4, Cary NC) to facilitate descriptive and inferential analyses. First, descriptive tests were conducted to obtain means (and SD) for all implementation outcome and determinant data, then split by experience level (0= inexperienced, 1= experienced). Independent samples t-tests were conducted to understand differences in implementation outcomes according to experience level. Pearson bivariate correlations were run to establish correlations between implementation outcomes and determinants to examine associations and to understand potential influences of implementation for schools that experienced greater success. All tests were run in SAS software, and α significance was assumed as p<.05, correlations with p<.10 were also highlighted due to the novel nature of this work. Following inferential testing, the research team explored qualitative extracts using NVivo as a means to contextualize findings from correlation analyses. Such an approach allowed for deeper contextual understanding of implementation practices which triangulate implementation determinants and outcomes (11).

Implementation Outcomes

Schools reported strong fidelity (mean score 7.6±2.91), however, this varied by schools and by item. Experienced schools reported better fidelity overall except for using the SWITCH website (see Figure 1). Parent outreach was the lowest implemented practice, outreach activities mostly entailed sending newsletters that were provided by the SWITCH team (70%), experienced schools reported more parent outreach practices than new schools. The most common method of school-wide integration was sending emails to the staff to inform them of the program and activities (88%) followed by using posters to promote SWITCH themes in different settings (73%). For adoption (5.53±2.17), experienced schools reported significantly higher rates according to independent samples t-tests (t=-2.03 p=0.04). This difference was consistent across use of modules, posters, and tracking/reinforcing themes. The highest implemented practice was classroom tracking followed by tracking in physical education setting (see Additional File 7).

Regarding penetration, behavioral tracking data demonstrate that new and experienced schools were approximately equal in terms of tracking rates between week 1 and week 7, 43±29% of students in new schools and 46±32% of returning schools tracked each week (mean score 0.448, or 45%). Tracking naturally dropped due to COVID-19-related school closures but it is noteworthy that rates were essentially 0% for new schools but 25% for experienced schools. This indicates that the returning schools were more likely to retain tracking rates to a greater extent than new schools. Only data from the first 8 weeks are used for the related correlation analyses.

Implementation Determinants

The process of converting qualitative interview data to numerical scores through CFIR protocols facilitated our ability to detect factors that were influential to SWITCH implementation outcomes. However, analysis of Cronbach’s alpha revealed that none of the CFIR domains had acceptable internal consistency (all <.40). We therefore felt it important to show variability in the data as opposed to means and SD of global domains. Figure 2 displays scores from each domain as dual-sided histograms to facilitate examination of variability, separated by experience level. From examination it appears that for all schools, factors within the Outer Setting and Implementation Process domains were more positively associated with implementation, but high variability must be noted. Experienced schools reported more positive influence from Readiness and Inner Setting domains compared to new schools who scored lower in these areas.

Table 2 displays all means ± SD for CFIR construct data. In terms of positive influential factors, data reveal that the most positive scores from coding of interview data were Readiness for Implementation – Leadership Engagement (i.e., building administration involvement/support, mean= 1.22±1.02), Individual Characteristics – Knowledge and Beliefs about the Intervention (i.e., school wellness teams’ perceptions of SWITCH, 1.51 ± 0.51), and Implementation Process – External Stakeholders (i.e., county 4-H Extension officer support, 1.42 ± 1.12). Regarding negative influences, lowest scores were assigned to Inner Setting – Relative Priority (i.e., priority given to SWITCH over other programs, -0.31 ± 1.26), Readiness for Implementation – Available Resources (i.e., time, personnel, equipment, -0.96±0.88), and challenges in Implementation Process - Key Stakeholders (i.e., engaging parents, -0.22 ± 1.31).

Relationships between Outcomes and Determinants – Whole Sample

Table 3 illustrates the results from Pearson bivariate correlation analyses for the whole sample and subsamples (new and experienced- discussed below). Almost all associations were positive, except Inner Setting-Networks and Communications (r=-.28, p=.07) and Tension for Change (r=.27, p=.09), both negatively correlated with Adoption. Tension for Change was also negatively associated with Penetration (r=-.33, p=.02). Table 4 illustrates the specific CFIR domains and constructs and their relation to specific implementation outcomes via qualitative data of corresponding quotes from interviews. In regard to positive associations with implementation, most of these relationships emerged from the Inner Setting and Implementation Process Domains.

Figure 3 displays findings from the SWRA tool to assess baseline readiness/capacity. For the overall sample, a significant correlation was found between classroom readiness and adoption (r=.366, p=.02). This illustrates that schools that reported greater classroom capacity were also using modules, tracking, and using posters more often than schools with lower classroom capacity. For the new schools, overall school capacity was positively correlated to adoption, indicating that organization-level readiness was associated with use of best practices across the school (r=.513, p=.04). The lack of relationship between other capacity indicators and implementation outcomes is potentially due to the lack of variability in the capacity means. School and Class capacity had the largest range in scores (2 or 2.25 to 5) compared to PE and lunch capacity (between 3 and 5).

Differences between New and Experienced Schools

Unique barriers to new schools comprised Relative Advantage, negatively associated with penetration (r=-.43, p=.08), Engaging Key Stakeholders (i.e., parents, r=.54, p=.02), and Reflecting and Evaluating (process) constructs (r=.63, p=.007). When asked about how they engaged parents, one school representative explained:

I think one of our goals was reaching out to parents more, and I think we were better. Especially when we put the activities into the family fun nights, and sending home occasional thing with the parent newsletter. Are we perfect at that yet? No. But you just get so consumed with everything else.

Some school staff members felt they were taking on too much in SWITCH, and engaging parents was not a priority: “I think, especially with this being our first year in SWITCH, it was just hard for us to figure out how to involve parents when we were still trying to figure out the program ourselves.” These extracts indicate that new schools needed more time to get comfortable with SWITCH, and engaging parents may not have been treated with the same priority as making healthy changes to the school environment.

Regarding Reflecting and Evaluating, some new schools found it particularly challenging to make time to meet as a team and evaluate progress: “If we would have [set] little [goals], and then each week get together, I think I could have done a better job at connecting the three of us ... I think that [is] where we have lost a little bit of steam.” Finally, fidelity was negatively associated with Outer Setting – Student Needs and Resources (r=-.47, p=.06). These findings indicate areas which presented barriers for implementation in new schools that had no prior experience with SWITCH. No unique positive associations were found in this sample.

For experienced schools, there were some unique positive determinants that did not emerge when examining the whole sample or new schools. First, Evidence Strength and Quality was positively associated with Fidelity (r=.40, p=.05), which did not appear for the whole sample of new schools. One experienced school team stated that experience in the program allowed school staff to see its effectiveness on the school environment: “So the first few years we had moaners and groaners ... they want[ed] to see it's going to work before they jump on ship with you. So each year we do it I think it gets better and better.” This perspective was shared by another school representative who stated:

When I read the program and saw that it was teaching life skills, to me that is the most invaluable education we can give to our students. And then I think this year, seeing the changes that had been made to get a little more support, so that it wasn't all of us creating on our own, that excited me again.

A consistent facilitator for all schools was Engaging (general, various stakeholders), but for experienced schools these constructs were associated with two implementation outcomes (Penetration r=.46, p=.01, Fidelity r=.53, p=.007). Finally, the Implementation Process domain highlighted that Engaging Implementation Leaders was significantly related to penetration (r=.37, p=.05), and Key Stakeholders was associated with Fidelity (r=.40, p=.05). Returning schools talked at length about their experiences working together as a leadership team, which is exemplified by one school:

So, our fifth grade teacher [is] on the SWITCH team as well and she went above and beyond what the rest of us did. She would put things out on social media quite a bit. She and I would both do that, but she also was the one that she printed off the parent letters and everything to get those ready to go. And she gave those to staff members to hand out to their kids.

The majority of school wellness teams adopted a similar approach whereby one member was the lead, accompanied by others who played a vital support role. This ensured a smooth implementation process. Although weakly related, Engaging Key Stakeholders (parents) seemed to be slightly more discussed with experienced schools. One school team member described their communication methods: “At the beginning, it was weekly emails that we were sending and lesson ideas to families, and so we're including the weekly newsletters of activities with those.” One school talked about how they struggled to send direct communications to parents due to school policy, but were able to engage them at a parent-teacher conference:

The night we did parent-teacher conferences, we got that blender bike, and I was amazed at how many parents stopped and talked to us and asked us more questions. So I think that the family aspect is there. I just don't think the school does much of that outside of just that parent-teacher conferences.

These examples illustrate the creativity and effort that school wellness teams put into SWITCH implementation, and the various ways they engaged parents as stakeholders.

The purpose of this study was to investigate the relationships between implementation determinants and outcomes of SWITCH, in addition to the nuanced determinants for new and experienced schools, with the goal of enhancing sustainability and facilitating scale-up efforts. The use of CFIR as a guiding framework is novel in the school wellness setting, specifically the use of systematic data analysis procedures, which facilitated a deep contextual understanding of relationships between implementation determinants and outcomes. Thus, a key innovation is the adaptation of a framework predominantly intended for healthcare settings (i.e., CFIR) to the school setting, marking an important advancement in the field of implementation science.

The SWITCH program represents a capacity-building process which allows school wellness teams to develop and sustain comprehensive programs of their own which in turn are more sustainable over time. The moderate-high rates of Penetration also correspond with self-reported Adoption of program best practices across the school setting. Implementation data from adoption, fidelity, and penetration measures highlight the differences between new and returning schools, a result that aligns with preliminary findings from prior evaluations (11). However, the finding that all schools struggled to engage parents despite increased efforts in the 2020 academic year reflects a wealth of prior research documenting this lack of engagement problem (8, 39, 40). Outreach practices of sending communications (emails/newsletters) and holding events for parental engagement were the most frequently reported, reflecting similar trends with school nutrition program promotion (41). Such findings stress the need to view implementation outcomes as incrementally changing constructs that must be studied over time, data from which can inform implementation strategies and eventually a continuous quality improvement model for school wellness integration (33, 42).

The finding that perceived Innovation Source, Complexity, and Cost were related to fidelity and adoption are not surprising, and reflect prior research conducted with Veterans Association clinics implementing a healthy weight intervention (26). Specifically, Damschroder et al., found that in addition to these factors, perceived Relative Advantage of the intervention was significantly different between high and low implementing sites, indicating that implementation strategies should be developed to address this barrier. Findings from interview data support the notion that schools generally perceive SWITCH to be a high-quality, feasible program, which facilitated buy-in from other school staff and stakeholders, a positive finding and one that will guide dissemination efforts.

The finding that outer setting factors of Student Needs and Resources and Cosmopolitanism were related to fidelity and penetration, respectively, was not surprising, but encouraging that a large number of schools reported that these were enabling factors to implementation. Having links to other schools and organizations was viewed as a positive determinant of fidelity, interview data yielded some reasons for this, such as implementation support for delivering lessons and additional program materials and equipment, which may have further pushed a culture of health in school buildings. Although some initial research has demonstrated the positive role of external networks and support (43, 44), very little is known about the effectiveness of implementation strategies which provide targeted support from this domain. Accordingly, a potential implementation strategy for future work with schools may be to provide a local network of support, bringing together other sectors such as food retail and community centers, ultimately enhancing the culture of health in the community (45, 46).

Across the whole sample, the inner setting factors of Tension for Change and Networks and Communications were negatively associated with implementation outcomes. Despite positive findings regarding Student Needs and Resources (outer setting), schools reported challenges when ensuring scheduled time for students to engage in SWITCH programming. The physical education and lunchroom settings were cited numerous times as areas for needed improvement and communication, such findings align with prior research on SWITCH and other comprehensive health promotion interventions and policies (11, 20, 47-49). As such, it seems that school stakeholders agreed on the “need” for school health and wellness promotion, but school-level schedules and policies did not seem to reflect this mission.

Within the D&I literature, the Expert Recommendations for Implementing Change (ERIC) research provides groundwork for selecting implementation strategies based on reported implementation challenges through models such as CFIR, facilitating tailored implementation support (50-52). For example, a CFIR-ERIC matching protocol conducted by Waltz et al., (52) highlighted that for Tension for Change implementation barriers, experts recommended “identify and prepare champions” and “conduct local needs assessment” as potential implementation strategies. In SWITCH, a core wellness team of at least three school staff members are trained over the course of a semester and complete the SWRA tool and School Wellness Environment Profile assessment, thus these strategies are already key components of the intervention model. As such, strategies within ERIC, developed intentionally for healthcare research, may not translate well to school and community settings without careful adaptation. This gap highlights the need for replication of the Waltz et al. study and others with this population, in order to advance the quality of school-based D&I research.

As expected, the Readiness for Implementation domain and findings from the SRWA assessment highlight the importance of capacity-building programs for systems change (9). The finding that Leadership Engagement and school-level capacity were positively related to implementation across both samples indicates that continued engagement of school administration is essential to SWITCH. A commonly reported barrier in school interventions is the lack of support or involvement from school administration (47, 53-55), thus a key strength of SWITCH is that administrators were able to be a part of the wellness team and attend conferences and trainings, enhancing their exposure to—and awareness of—school wellness programming. Two constructs that highlighted differences between new and returning schools were Learning Climate and Available Resources, with experienced schools reporting positive associations with implementation, whereas new schools reported no influence or negative (non-significant) influence on implementation. Examples from interviews highlighted the role of personnel time, equipment availability, and funding as supports for implementation. Therefore, an implementation strategy for new schools may be a cost-matching initiative through local county 4-H extension or through collaborating with community stakeholders, as described above and recommended through findings of the Waltz et al., (52) Such initiative would potentially provide an equitable and cost-effective implementation strategy for new schools and further enhancing cross-sector collaborations to promote school and community health (46).

Finally, the Implementation Process domain revealed the most differences between new and experienced schools. Mutual constructs positively related with implementation were Engaging – General, Opinion Leaders, and Innovation Participants. Related to innovation participants, youth advocacy in school wellness and health promotion has been demonstrated as an effective strategy for implementation and student health outcomes (56-58) and some studies are emerging regarding how student advocacy groups can be studied through a D&I lens (59). Engaging – Key Stakeholders and Reflecting & Evaluating were seen as negative implementation determinants for new schools, whereas for experienced schools there were positive relationships. Parents have been reported as the most difficult stakeholder group to engage in school-wide initiatives, and in previous cycles of SWITCH (11, 28, 39), however, some schools reported that when they did hold an event at school or at another academic-related event (i.e., parent-teacher conference), parents showed great interest. The most commonly cited outreach practice was disseminating newsletters (developed by SWITCH team) to parents, with events held less frequently. Thus, more research is needed to identify effective ways for engaging parents in school wellness, ideally with parents as the primary participants, to identify potential implementation strategies. Further, although the SWITCH team comprised reports based on checkpoint surveys (adoption and fidelity) to be disseminated to schools, it seems new schools reported lack of evaluation and reflection. As such, a future implementation strategy may be to provide tailored support to schools (through online modules or other easily accessible formats) regarding how to interpret their performance and set goals for implementation, based on expert recommendations (51, 52).

Limitations

There are some limitations that could influence interpretation from this type of evaluation. First, and most important, the COVID-19 pandemic-related school closures prevented completion of the 12-week implementation cycle, thus it is not clear whether differences between new and experienced schools would have persisted or varied. Further, to prevent overburdening school staff, we refrained from collecting checkpoint survey data once schools closed and began remote learning, which may have limited understanding of fidelity and adoption within schools. Finally, this project is the first of its kind in that the CFIR qualitative analysis and systematic interview coding methods were tailored to non-healthcare settings. Thus, our interpretations of the CFIR constructs/methods and their applications to school and community-based settings may need to evolve over time as replication of these methods occur. Ongoing work with SWITCH has utilized these findings but the results provide generalizable insights about factors that influence the scale up of interventions (60). The process and systematic approach to the use of CFIR in the analyses also provide a guide for other school-based researchers seeking to utilize D&I methods to evaluate programming.

The present study highlighted various determinants of implementation for SWITCH, which facilitate development of implementation strategies which can be tested and applied in future dissemination efforts. Specifically, it is clear that new schools to the program face greater challenges and need tailored support; findings further emphasize that SWITCH is a process to foster systems change in school settings and such process is ongoing. Mixed methods research is essential for enhancing our understanding of dissemination of evidence-based programming, and it provides invaluable context to identify why and how implementation efforts may succeed or be limited in effectiveness. The CFIR coding procedures provide an opportunity for school researchers and practitioners to utilize D&I frameworks and apply systematic evaluation methods which enhance the rigor and quality of implementation evaluations. A goal for school researchers, which is already emergent in mental health research, is to develop “common measures” for implementation determinants and outcomes (61–63); such work would enhance generalizability and transferability of qualitative findings to other contexts and geographic locations. What gets measured often is what gets achieved; sustainability is enhanced when the implementation supports are matched to school needs.

Ethics approval and consent to participate

Approval was obtained from the Institutional Review Board (#14–651) at Iowa State University to conduct the project. Participating schools completed an agreement form at the beginning of the 2019–2020 academic year in which they indicated their consent to participate in evaluation procedures. All school representatives were aged 18 and older.

Consent for publication

Not applicable

Availability of data and materials

Additional files provide information for all data collection and analysis procedures including the Standards for Reporting Implementation Studies: the StaRI checklist for completion (Additional File 8). The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request [email protected].

Competing interests

The authors have no competing interests to declare.

Funding

USDA NIFA grant: 2015–68001-23242. The USDA was not involved in the design of the study and collection, analysis, and interpretation of data or writing of the manuscript.

Authors' contributions

All authors contributed to the development of the research study and provided ongoing feedback throughout the implementation evaluation process. GMM and GJW led the multiple methods design and evaluation components, RS and GMM led the qualitative analysis procedures, RS and LL facilitated survey and interview data collection procedures, GMM analyzed survey data and developed measures for school capacity, RRR and JAL provided feedback on analysis and interpretation of qualitative data. All authors read and approved the final manuscript.

Acknowledgements

The authors wish to thank the School Wellness Teams (SWT) that participated in the intervention, led programming, and provided their feedback. We also wish to thank individuals who served on the broader evaluation team for the SWITCH grant and contributed to early conceptualization: David A. Dewaltoswki, Spyridoula Vazou, Lorraine Lanningham-Foster, and Douglas Gentile.

Authors' information (optional)

College of Public Health, Temple University, 1101 W. Montgomery Ave, Philadelphia, PA, 19122, USA

(Gabriella M. McLoughlin)

Implementation Science Center for Cancer Control and Prevention Research Center, Brown School, Washington University in St. Louis, One Brookings Drive, St. Louis, MO, 63130, USA

(Gabriella M. McLoughlin)

4-H Extension and Outreach, Iowa State University, Ames, IA, 50011, USA

(Rachel Sweeney and Laura Liechty)

Department of Health Sciences, University of Colorado Colorado Springs, Colorado Springs, CO, 80918, USA

(Joey A. Lee)

Department of Food, Nutrition, Dietetics and Health, Kansas State University, Manhattan, KS, 66506, USA

(Richard R. Rosenkranz)

Department of Kinesiology, Iowa State University, Ames, IA, 50011, USA

(Gregory J. Welk).

Mansfield JL, Savaiano DA. Effect of school wellness policies and the Healthy, Hunger-Free Kids Act on food-consumption behaviors of students, 2006-2016: A systematic review. Nutrition Reviews. 2017,75:533-52.
Watson A, Timperio A, Brown H, Best K, Hesketh KD. Effect of classroom-based physical activity interventions on academic and physical activity outcomes: a systematic review and meta-analysis. International Journal of Behavioral Nutrition and Physical Activity. 2017,14(1):114.
Sisnowski J, Street JM, Merlin T. Improving food environments and tackling obesity: A realist systematic review of the policy success of regulatory interventions targeting population nutrition. PLoS One. 2017,12(8):e0182581.
Russ LB, Webster CA, Beets MW, Phillips DS. Systematic review and meta-analysis of multi-component interventions through schools to increase physical activity. Journal of Physical Activity and Health. 2015,12(10):1436-46.
Amini M, Djazayery A, Majdzadeh R, Taghdisi M-H, Jazayeri S. Effect of school-based interventions to control childhood obesity: A review of reviews. International Journal of Preventive Medicine. 2015,6(68):1-15.
Brownson RC CG, Proctor EK. Future Issues in Dissemination and Implementation Research. In: Brownson RC CG, Proctor EK, editor. Dissemination and Implementation Research in Health: Translating Science to Practice. Second Edition ed. New York: Oxford University Press, 2018.
Brownson RC, Colditz GA, Proctor EK, editors. Dissemination and Implementation Research in Health: Translating Science to Practice, Second Edition. New York: Oxford University Press, 2018.
Herlitz L, MacIntyre H, Osborn T, Bonell C. The sustainability of public health interventions in schools: a systematic review. Implementation Science. 2020,15(1):4-.
Shoesmith A, Hall A, Wolfenden L, Shelton RC, Powell BJ, Brown H, et al. Barriers and facilitators influencing the sustainment of health behaviour interventions in schools and childcare services: a systematic review. Implementation Science. 2021,16(1).
McLoughlin GM, Vazou S, Liechty L, Torbert A, Lanningham-Foster L, Rosenkranz RR, et al. Transdisciplinary Approaches for the Dissemination of the SWITCH School Wellness Initiative Through a Distributed 4-H/Extension Network. Child & Youth Care Forum. 2020.
McLoughlin GM, Candal P, Vazou S, Lee JA, Dzewaltowski DA, Rosenkranz RR, et al. Evaluating the implementation of the SWITCH® school wellness intervention and capacity-building process through multiple methods. International Journal of Behavioral Nutrition and Physical Activity. 2020,17(1):162.
McLoughlin GM, Rosenkranz RR, Lee JA, Wolff MM, Chen S, Dzewaltowski DA, et al. The importance of self-monitoring for behavior change in youth: Findings from the SWITCH® school wellness feasibility study. International Journal of Environmental Research and Public Health. 2019,16(20):3806-.
Chen S, Dzewaltowski DA, Rosenkranz RR, Lanningham-Foster L, Vazou S, Gentile DA, et al. Feasibility study of the SWITCH implementation process for enhancing school wellness. BMC Public Health. 2018,18(1):1119-.
Welk GJ, Chen S, Nam YH, Weber TE. A formative evaluation of the SWITCH® obesity prevention program: print versus online programming. BMC obesity. 2015,2:20-.
Gentile DA, Welk GJ, Eisenmann JC, Reimer RA, Walsh DA, Rusell DW, et al. Evaluation of a multiple ecological level child obesity prevention program: Switch® what you Do, View, and Chew. BMC Medicine. 2009,7:49-.
Eisenmann JC, Gentile DA, Welk GJ, Callahan R, Strickland S, Walsh M, et al. SWITCH: rationale, design, and implementation of a community, school, and family-based intervention to modify behaviors related to childhood obesity. BMC Public Health. 2008,8(1):223-.
Lou Y, Wu H, Welk GJ, Vazou S, Chen S, Gentile DA, et al. The SWITCH Implementation Process on School Lunch Consumption Patterns and Plate Waste. Journal of Nutrition Education and Behavior. 2018,50(7):S167-S8.
Welk G, Chen S, Vazou S, Lanningham-Foster L, Gentile D, Rosenkranz R, et al. Implementation Feasibility of School Modules Designed to Enhance the Evidence-Based Switch Obesity Prevention Program. Journal of Nutrition Education and Behavior. 2016,48(7):S133.
Chen S, Liu Y, Welk G. Using a hybrid design to analyze effectiveness and implementation of a refined energy-balance education module for upper elementary physical education. Ejournal de la recherche sur l'intervention en éducation physique et sport -eJRIEPS. 2019(Hors-série N° 3).
Lee JA, McLoughlin GM, Welk GJ. School Wellness Environments: Perceptions Versus Realities. The Journal of School Nursing. 2020:1059840520924453.
Lee JA, Welk GJ, Vazou S, Ellingson LD, Lanningham-Foster L, Dixon P. Development and application of tools to assess elementary school wellness environments and readiness for wellness change (doctoral dissertation) 2018.
Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the Consolidated Framework for Implementation Research. Implementation Science. 2016,11(1):72-.
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009,4:50.
Fernandez ME, Walker TJ, Weiner BJ, Calo WA, Liang S, Risendal B, et al. Developing measures to assess constructs from the Inner Setting domain of the Consolidated Framework for Implementation Research. Implement Sci. 2018,13(1):52.
Soi C, Gimbel S, Chilundo B, Muchanga V, Matsinhe L, Sherr K. Human papillomavirus vaccine delivery in Mozambique: identification of implementation performance drivers using the Consolidated Framework for Implementation Research (CFIR). Implementation Science. 2018,13(1):151.
Damschroder LJ, Lowery JC. Evaluation of a large-scale weight management program using the consolidated framework for implementation research (CFIR). Implement Sci. 2013,8:51.
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Science. 2009,4(1):50-.
Leeman J, Wiecha JL, Vu M, Blitstein JL, Allgood S, Lee S, et al. School health implementation tools: a mixed methods evaluation of factors influencing their use. Implementation Science. 2018,13(1):48-.
Bozsik F, Berman M, Shook R, Summar S, DeWit E, Carlson J. Implementation contextual factors related to youth advocacy for healthy eating and active living. Translational Behavioral Medicine. 2018,8(5):696-705.
Cassar S, Salmon J, Timperio A, Naylor P-J, van Nassau F, Contardo Ayala AM, et al. Adoption, implementation and sustainability of school-based physical activity and sedentary behaviour interventions in real-world settings: a systematic review. International Journal of Behavioral Nutrition and Physical Activity. 2019,16(1):120-.
Koester M, Bejarano CM, Davis AM, Brownson RC, Kerner J, Sallis JF, et al. Implementation contextual factors related to community-based active travel to school interventions: a mixed methods interview study. Implementation Science Communications. 2021,2(1):94.
Roberts S, Keane J, Ward C, Restrick L. Plan, Do, Study, Act. Physiotherapy. 2002,88(12):769-.
Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011,38(2):65-76.
Holt DT, Helfrich CD, Hall CG, Weiner BJ. Are you ready? How health professionals can comprehensively conceptualize readiness for change. Journal of General Internal Medicine. 2010,25(SUPPL. 1):50-5.
Weiner BJ. A theory of organizational readiness for change. Implementation Science. 2009,4(1):67-.
Hawe P, Shiell A, Riley T. Theorising Interventions as Events in Systems. American Journal of Community Psychology. 2009,43(3-4):267-76.
Research CRT-CfCM. Consolidated Framework for Implementation Research- Qualitative Analysis.
Strauss A, Corbin J. Basics of qualitative research: Techniques and procedures for developing grounded theory. 4th ed. Thousand Oaks, CA: Sage Publications, 2015.
McDowall PS, Schaughency E. Elementary School Parent Engagement Efforts: Relations with Educator Perceptions and School Characteristics. Journal of Educational Research. 2017,110(4):348-.
Clarke JL, Griffin TL, Lancashire ER, Adab P, Parry JM, Pallan MJ. Parent and child perceptions of school-based obesity prevention in England: a qualitative study. BMC Public Health. 2015,15(1).
McLoughlin GM, Turner L, Leider J, Piekarz-Porter E, Chriqui JF. Assessing the Relationship between District and State Policies and School Nutrition Promotion-Related Practices in the United States. Nutrients. 2020,12(8):2356.
Brownson RC, Colditz GA, Proctor EK. Dissemination and implementation research in health: translating science to practice: Oxford University Press, 2012.
Watson DP, Adams EL, Shue S, Coates H, McGuire A, Chesher J, et al. Defining the external implementation context: an integrative systematic literature review. BMC Health Serv Res. 2018,18(1):209.
Weatherson KA, Gainforth HL, Jung ME. A theoretical analysis of the barriers and facilitators to the implementation of school-based physical activity policies in Canada: a mixed methods scoping review. Implement Sci. 2017,12(1):41.
Mazzucca S, Arredondo EM, Hoelscher DM, Haire-Joshu D, Tabak RG, Kumanyika SK, et al. Expanding Implementation Research to Prevent Chronic Diseases in Community Settings. Annual Review of Public Health. 2021.
Kumanyika SK. A Framework for Increasing Equity Impact in Obesity Prevention. American Journal of Public Health. 2019,109(10):1350-7.
McLoughlin GM, Graber KC, Woods AM, Templin T, Metzler M, Khan NA. The Status of Physical Education Within a Nationally Recognized School Health and Wellness Program. Journal of Teaching in Physical Education. 2020,39(2):274-83.
Levay AV, Chapman GE, Seed B, Wittman H. District-level implementation of British Columbia's school food and beverage sales policy: a realist evaluation exploring intervention mechanisms in urban and rural contexts. Can J Public Health. 2019,110(1):21-30.
McIsaac JD, Penney TL, Masse L, Kirk SFL. The Association between Perceived Adequacy and Capacity for School Food Policy Implementation with Food Availability and Policy Adherence in Nova Scotia, Canada. Int J Environ Res Public Health. 2019,16(11).
Powell B, Beidas R, Lewis C, Aarons G, McMillen J, Proctor E, et al. Methods to Improve the Selection and Tailoring of Implementation Strategies. The Journal of Behavioral Health Services & Research. 2017,44(2):177-94.
Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015,10:21.
Waltz TJ, Powell BJ, Fernández ME, Abadie B, Damschroder LJ. Choosing implementation strategies to address contextual barriers: diversity in recommendations and future directions. Implementation Science. 2019,14(1):42.
Levay AV, Chapman GE, Seed B, Wittman H. Examining school-level implementation of British Columbia, Canada's school food and beverage sales policy: a realist evaluation. Public Health Nutr. 2020,23(8):1460-71.
Carlson JA, Engelberg JK, Cain KL, Conway TL, Geremia C, Bonilla E, et al. Contextual factors related to implementation of classroom physical activity breaks. Translational Behavioral Medicine. 2017,7(3):581-92.
Allison KR, Vu-Nguyen K, Ng B, Schoueri-Mychasiw N, Dwyer JJM, Manson H, et al. Evaluation of daily physical activity (DPA) policy implementation in Ontario: Surveys of elementary school administrators and teachers. BMC Public Health. 2016,16(1):1-16.
Millstein RA, Woodruff SI, Linton LS, Edwards CC, Sallis JF. Development of measures to evaluate youth advocacy for obesity prevention. Int J Behav Nutr Phys Act. 2016,13:84.
Millstein RA, Woodruff SI, Linton LS, Edwards CC, Sallis JF. A pilot study evaluating the effects of a youth advocacy program on youth readiness to advocate for environment and policy changes for obesity prevention. Transl Behav Med. 2016,6(4):648-58.
Morse LL, Allensworth DD. Placing students at the center: The whole school, whole community, whole child model. Journal of School Health. 2015,85:785-94.
Lane HG, Deitch R, Wang Y, Black MM, Dunton GF, Aldoory L, et al. "Wellness Champions for Change," a multi-level intervention to improve school-level implementation of local wellness policies: Study protocol for a cluster randomized trial. Contemporary Clinical Trials. 2018,75:29-39.
McKay H, Naylor P-J, Lau E, Gray SM, Wolfenden L, Milat A, et al. Implementation and scale-up of physical activity and behavioural nutrition interventions: an evaluation roadmap. International Journal of Behavioral Nutrition and Physical Activity. 2019,16(1):102-.
Locke J, Lee K, Cook CR, Frederick L, Vázquez-Colón C, Ehrhart MG, et al. Understanding the Organizational Implementation Context of Schools: A Qualitative Study of School District Administrators, Principals, and Teachers. School Ment Health. 2019,11(3):379-99.
Lyon AR, Cook CR, Brown EC, Locke J, Davis C, Ehrhart M, et al. Assessing organizational implementation context in the education sector: confirmatory factor analysis of measures of implementation leadership, climate, and citizenship. Implementation Science. 2018,13(1):5.
Lyon A, Frazier S, Mehta T, Atkins M, Weisbach J. Easier said than done: Intervention sustainability in an urban after-school program. Administration & Policy in Mental Health & Mental Health Services Research. 2011,38:504-17.

Table 1: School Demographic Information for the 2019–2020 Cohort

	Free/Reduced Meals (%)		Racial/Ethnic Minority (%)		Enrollment		Experience (years)
	Mean	SD	Mean	SD	Mean	SD	Mean	SD
Total (n=52)	49.1	19.0	15.4	18.6	226.3	180.1	1.8	0.8
New (n=22)	50.7	21.9	20.0	21.5	224.6	200.5	0	0
Experienced (n=30)	48.0	17.2	12.1	15.9	227.5	168.0	0	0

Note: Free/reduced meals = percentage of students eligible for free/reduced meals, experience = years of experience in the program including present year (range 1–4)

Table 2: CFIR Coding Results by School Experience

CFIR Domain	Construct	New		Experienced		Total
		Mean	SD	Mean	SD	Mean	SD
Intervention Characteristics	Innovation Source	0.41	0.71	0.21	0.57	0.29	0.63
	Evidence Strength and Quality	1.06	0.90	0.79	0.88	0.89	0.88
	Relative Advantage	0.12	0.49	0.11	0.42	0.11	0.44
	Adaptability	0.71	0.69	0.96	0.88	0.87	0.81
	Trialability	0.00	0.00	0.07	0.66	0.04	0.52
	Complexity (reverse)	0.06	1.03	0.43	0.92	0.29	0.97
	Design Quality	0.71	0.85	0.25	1.27	0.42	1.14
	Cost (reverse)	0.41	0.51	0.64	0.87	0.56	0.76
Outer Setting	Student Needs and Resources	0.41	0.71	0.75	0.84	0.62	0.81
	Cosmopolitanism	0.53	1.23	0.93	1.18	0.78	1.20
	Peer Pressure	0.71	1.05	0.25	0.70	0.42	0.87
	External Policy and Incentives	1.06	1.14	0.68	0.94	0.82	1.03
Inner Setting	Structural Characteristics	0.00	1.00	0.39	1.23	0.24	1.15
	Networks Communications	0.53	1.37	0.43	1.26	0.47	1.29
	Tension for Change	0.29	0.59	0.32	0.61	0.31	0.60
	Relative Priority	-0.41	0.94	-0.25	1.43	-0.31	1.26
	Culture	0.88	0.86	0.82	0.94	0.84	0.90
	Compatibility	0.76	1.15	1.32	0.67	1.11	0.91
	Organizational Incentives and Rewards	0.00	0.00	0.07	0.26	0.04	0.21
	Goals and Feedback	0.53	0.80	0.43	0.57	0.47	0.66
Readiness for Implementation	Readiness for Implementation	-0.12	1.22	0.89	0.74	0.51	1.06
	Learning Climate	-0.12	0.33	0.14	0.52	0.04	0.47
	Leadership Engagement	1.35	1.06	1.14	1.01	1.22	1.02
	Available Resources	-1.24	0.83	-0.79	0.88	-0.96	0.88
	Access to Knowledge and Information	0.65	1.46	0.36	1.16	0.47	1.27
Individual Characteristics	Knowledge and Beliefs about Intervention	1.35	0.49	1.61	0.50	1.51	0.51
	Self-Efficacy	-0.24	1.35	0.61	0.92	0.29	1.16
	Individual Stage of Change	-0.06	0.24	-0.07	0.54	-0.07	0.45
	Individual Identification with Organization	0.00	0.00	-0.07	0.66	-0.04	0.52
	Other Personal Attributes	0.41	1.12	0.25	1.08	0.31	1.08
Implementation Process	Planning	0.29	1.05	0.21	1.42	0.24	1.28
	Implementation Leaders	0.88	1.11	0.79	0.74	0.82	0.89
	Engaging	0.47	1.50	1.00	1.15	0.80	1.31
	Opinion Leader	0.24	1.39	0.54	1.07	0.42	1.20
	Champions	0.94	0.90	1.07	1.02	1.02	0.97
	External Change Agents	1.24	1.15	1.54	1.10	1.42	1.12
	Key Stakeholders	-0.47	1.18	-0.07	1.39	-0.22	1.31
	Innovation Participants	1.24	1.09	1.75	0.44	1.56	0.78
	Executing	0.59	1.12	0.86	1.01	0.76	1.05
	Reflecting and Evaluating	0.12	0.33	0.25	0.65	0.20	0.55

Note: CFIR = Consolidated Framework for Implementation Research, reverse = items reverse coded, all items scored on a range from -2 to +2, for all definitions please see coding consensus document- supplementary file XX.

Table 3: Pearson bivariate correlations between implementation determinants and outcomes

CFIR Domain	Construct	All Schools (n=47)			New Schools (n=17)			Experienced Schools (n=30)
		Penetration	Fidelity	Adoption	Penetration	Fidelity	Adoption	Penetration	Fidelity	Adoption
Intervention Characteristics	Innovation Source	0.05	0.43**	-0.110	0.18	0.62**	0.160	-0.03	0.37*	-0.27
	Evidence Strength and Quality	-0.03	0.13	0.09	-0.28	-0.18	0.43*	0.11	0.40**	-0.06
	Relative Advantage	0.02	-0.04	-0.04	-0.43*	-0.33	-0.55*	0.30	0.18	0.34
	Adaptability	0.15	0.03	-0.05	-0.21	0.1	0.03	0.30	-0.04	-0.14
	Trialability	0.05	0.26	-0.11	NA	NA	NA	0.06	0.35	-0.14
	Complexity (reverse)	0.07	0.32**	0.08	-0.24	0.36	0.23	0.26	0.21	-0.15
	Design Quality	0	0.02	0.09	0.16	-0.17	0.39	-0.05	0.20	0.07
	Cost (reverse)	0.22	0.08	0.49**	0.29	0.08	0.16	0.20	0.01	0.59**
Outer Setting	Student Needs and Resources	0.26*	-0.04	0.19	0.18	-0.47*	0.09	0.30	0.09	0.15
	Cosmopolitanism	0.2	0.26*	0.05	0.22	-0.18	-0.36	0.20	0.51**	0.25
	Peer Pressure	0.09	-0.03	0.08	0.13	-0.01	0.15	0.08	0.08	0.19
	External Policy and Incentives	0.05	0.22	-0.25	-0.21	0.3	-0.1	0.22	0.26	-0.29
Inner Setting	Structural Characteristics	0.14	0.22	-0.02	-0.16	0.13	0.08	0.26	0.22	-0.13
	Networks Communications	0.03	0.17	-0.28*	-0.18	0.21	-0.27	0.16	0.15	-0.31
	Culture	0.39**	0.48**	0.27*	0.03	0.66**	0.03	0.57**	0.38	0.43
	Tension for Change	-0.33**	-0.16	-0.27*	-0.55	-0.08	0.18	-0.22	-0.24	-0.59 **
	Compatibility	-0.1	0.02	-0.06	-0.27	-0.33	-0.44*	0.03	0.26	0.15
	Relative Priority	0.31**	0.34**	0.12	0.3	0.52**	-0.03	0.32*	0.29	0.18
	Organizational Incentives and Rewards	0.31**	0.18	0.01	NA	NA	NA	0.38**	0.19	-0.06
	Goals and Feedback	0.05	0	-0.24	-0.09	-0.24	-0.22	0.15	0.26	-0.25
Readiness for Implementation	Learning Climate	0.11	0.31**	0.34**	-0.21	0.22	0.03	0.22	0.31	0.39*
	Readiness for Implementation	0.06	0.21	-0.03	0.13	0.13	-0.11	0.00	0.10	-0.25
	Leadership Engagement	0.31**	0.44**	0.42**	0.25	0.31	0.51**	0.35*	0.59**	0.42**
	Available Resources	0.28*	0.32**	0.01	0.18	-0.17	-0.27	0.34*	0.56**	0.06
	Access to Knowledge and Information	-0.19	-0.2	-0.133	-0.22	-0.39	0.13	-0.18	-0.02	-0.32
Individual Characteristics	Knowledge and Beliefs about Intervention	0.01	0.18	0.21	-0.22	-0.06	0.29	0.12	0.24	0.04
	Self-Efficacy	0.05	0.33**	-0.01	0.12	0.34	-0.29	-0.00	0.25	0.07
	Individual Stage of Change	-0.02	0.16	0.47**	-0.04	-0.18	-0.01	-0.02	0.29	0.67**
	Individual Identification with Organization	0.35**	0.13	-0.07	NA	NA	NA	0.43**	0.17	-0.10
	Other Personal Attributes	0.08	0.33**	-0.14	-0.3	0.08	-0.22	0.29	0.56**	-0.06
Implementation Process	Planning	0.18	0.50**	0.27*	-0.1	0.32	0.24	0.28	0.63**	0.32
	Engaging	0.23	0.54**	0.2	-0.11	0.51**	0.1	0.47**	0.53**	0.19
	Opinion Leader	0.05	0.59**	0.19	-0.14	0.52**	0.36	0.17	0.65**	0.04
	Implementation Leaders (SWT)	0.033	0.2	0.37**	-0.38	0.12	0.39	0.37**	0.33	0.44**
	Champions	0.21	0.42**	0.32**	0.14	0.25	0.34	0.24	0.52**	0.29
	External Change Agents	0.08	0.31*	-0.19	0.22	0.23	-0.42	0.01	0.32	-0.11
	Key Stakeholders	-0.06	0.15	-0.06	-0.54**	-0.33	-0.25	0.16	0.40*	-0.02
	Innovation Participants	0.2	0.44**	0.11	0.27	0.43*	0.21	0.18	0.45**	-0.22
	Executing	0.38**	0.40**	0.02	0.48**	0.19	-0.12	0.32*	0.53**	0.05
	Reflecting and Evaluating	0.12	0.33**	0.39**	-0.63**	-0.03	0.46*	0.33*	0.44**	0.36*

Note: NA = not applicable due to lack of data to run correlations, * = p<.1, **= p<.05

Table 4: Qualitative extracts aligning with relationships from interview data

Domain	Construct	Relationship with outcome (+/-)	Interview Extract
Intervention Characteristics	Innovation Source	Fidelity (+)	“I think it helps that it's not [wellness team] saying, ‘You guys should do this.’ We're bringing a program that's research based through the Extension office, so there's great buy in, knowing that this has been practiced before, it's been successful in eastern Iowa, and we're not just asking them to hop on any old thing we found online. So I feel like they were totally on board.”
	Complexity	Fidelity (+)	“At least for me, the elementary part was pretty straightforward. Middle school, kind of the same. At least for middle school, how to get everyone somewhat like, ‘Hey, how do we make them stay on track and just try and make sure we reach everyone?’ That's the most, I guess, complicated part of it.”
	Cost	Adoption (+)	“It doesn't cost us any more to include everybody, than what it is. So that's a great benefit to the whole school that nobody's left out. Sometimes pre-k doesn't do it, but everybody's involved, which helps us tremendously.”
Outer Setting	Student Needs and Resources	Penetration (+)	“So [students] had so much fun and really the junior high teachers said, "I wish they could realize how much better behaved they were after a fun recess when they were moving around and giggling and letting off that stress by giggling and moving than when they just sit there and gossip." That's about all girls would do normally if you didn't have an activity going. So that I thought was great. We had more teachers involved to do all that. That was really kind of fun.”
	Cosmopolitanism	Fidelity (+)	“We did a lot with 5-2-1-0 also this year [and] SWITCH because we became a 5-2-1-0 site, as well. And so, we did a lot of that with SWITCH at the same time, so that really made it good to just incorporate all that together.”
Inner Setting	Tension for Change	Penetration (-) Adoption (-)	“Also, we always say... We had some teachers that are like, "Oh, I need to keep this kid for PE." Like, I don't think so, unless he's coming in some other time to do it. So I mean, I think they're starting to understand that the kids need to be in there, that they need that [active] time.”
	Relative Priority	Fidelity (+) Penetration (+)	“I don't know that I would say it really took a backseat. I don't know that there was other high priority issues. There were things that we had to do, yes, but I think it was just an equal like, ‘You got to do that. One way or another, we'll get done, and it's going to happen.’ ”
	Culture	Fidelity (+)	“I think the supportive atmosphere helped, that everyone could get on board and see that this was important and how we could work together to do it and help each other out.”
	Networks & Communications	Adoption (-)	“The PE teacher, I had no interaction with. So I guess, next year my goal would be just to get those teachers early in the fall to buy into it a little bit more.”
Readiness for Implementation	Learning Climate	Adoption (+) Fidelity (+)	“I think between the two grade levels being able to work well together, it was easy to ask each other questions or even sometimes remind each other, ‘Hey, don't forget to log your kids' data for the week and things like that. Without hurting feelings, but just making sure that we're a team. So what we do, we do it together, and we do it well together.”
	Leadership Engagement	Adoption (+) Fidelity (+) Penetration (+)	“I know our principal is really on board. She came to the conference at the beginning of the year. She really likes the program, as well as all of the ideas that it covers.” “They were total, our principal was totally backing it. She was really embracing it. Before I had presented this to her, she was already working on a sensory pathway, getting that ordered and put together. So, she was already on top of wanting to increase movement and activity.”
	Available Resources	Penetration (+) Fidelity (+)	“I love that it's not a huge amount of time, but it does take time to get it implemented.” “I know that resources are scarce but I have everything that I need to teach. I have the latest curriculum. I get a say in what we teach, which is nice… We have a big fundraiser and they were asking teachers to put in things for their wishlist. What do you need? What do you want? And I'm hard-pressed to say I need anything more, at least right now.”
	Self-Efficacy	Fidelity (+)	“Fairly confident. Worried about my first year and getting everything in, but, yeah, everyone was really helpful in sending out materials and asking if we had questions, and they were always there to answer them on time.”
	Individual Stage of Change	Adoption (+)	“I would say it's very helpful, for me personally, to bring about diversity into your lesson. I have all kinds of plans of getting around and moving. One of my goals, and I don't meet it, and that makes me mad because it's a high goal that I have, is that I want to make learning more active.”
Individual Characteristics	Other Personal Attributes	Fidelity (+)	“But again, it comes down to the teacher who implemented it within her classroom. She was the one who ... And again, you have these people on all of your teams, you have the people who you can't put one more thing on their plate, and then you have others who are like, ‘Oh my gosh, I don't care what it takes. I'll be up to midnight. Let's do it.’”
Implementation Process	Planning	Fidelity (+)	“I was confident. I felt well going into it. The head of the dietary program and I were the ones that went to the training. I thought that went really well and felt comfortable leaving and ready to start it up again.” “You guys had a document that was to use as a guide, and then we also used that note taking form, and we wrote our notes in there. I kept the notes, and then every team meeting I'd bring the notes with us.”
	Engaging	Fidelity (+)	“Involving the students really helps a lot. Some of the students who were really excited about it and the teachers see that those students are excited about it and then also keeping it simple for some of those teachers because too much, they're just going to not want to do it.” “Every time I would get something I would try to send that out. I did go every week and make sure we got the trinkets sent out. At the beginning, remember, I guess I went in and I gave a 20 minute presentation on how to do the website for the kids.”
	Opinion Leader	Fidelity (+)	“The classroom teachers did a pretty nice job of trying to implement some things. They really took it on, but it really was kind of big.” “We're both classroom teachers so our role was just implementing it within the classroom and supporting the other classroom teachers if they needed help. One of our other teachers had been through some of the training the year before, so she also was pretty helpful.”
	Implementation Leaders	Adoption (+)	“When we look at the book, it's pretty straightforward on that. It's just finding the time to have, especially with the classroom teachers, to have them help implement and reinforce what we're teaching. We gave them posters and stuff.” “I'm [leader], I'm the school nurse. I took coordination aspect of SWITCH and then recruited both [teachers] to help me with that. The three of us pretty much worked together. We did invite all the other teachers anytime we met, so that we could have everybody around the table for all the decision making.
	Champions	Fidelity (+) Adoption (+)	“So, our ‘at risk’ [counselor] and our guidance counselor were really curious to what the kids were logging in on because they were talking about it in middle school. And so, they were the ones, the first two that actually offered with family night that was scheduled to help out with anything with that too, and the SWITCH programs.”
	Innovation Participants	Fidelity (+)	“We were starting to get results, better results with the students. Like they would report to me like, ‘Oh, I didn't use my phone at all.’ Or, ‘I'm only going to use it up until this certain hour.’ Like they had, they were able to vocalize their goals and report back on their goals a lot easier.” “It's just I think, now, really, since we've done it three years, the kids just expect that we're doing it. Because that first year we bought pedometers, and they had a walking goal, and the last two years, I have parents saying, ‘Oh, the kids can't wait until they get to wear those pedometers.’ They think that's just part of what they're supposed to do.”
	Executing	Fidelity (+) Penetration (+)	“One of the things that we have to teach, like just switching what you ‘do’ part, I'm talking about sedentary lifestyle versus an active lifestyle and what that looks like in the short-term and long-term within their lifespan. It flows perfectly within health class.” “And so, of course, implementation is pretty seamless because my staff know about the program, they know what to do now, they know what kind of data to take, those sorts of things. So, as far as implementation, from my standpoint, it was very good.”
	Reflecting and Evaluating	Adoption (+) Fidelity (+)	“We tried the folders this year with a specific lesson. She asked how that was going. I said, ‘Well, sometimes another teacher needs it at the same time.’ So then we used that feedback. She made two folders so two teachers could have the same lesson at the same time.” “We keep making changes. We learn from our things and then we fix it and do it again, so I think we've done a good job. We found kind of what works in our school.”

SupplementaryFiles.docx

Evaluation of a Large-Scale School Wellness Intervention Through the Consolidated Framework for Implementation Research (CFIR): Implications for Dissemination and Sustainability

Status:

Version 2

Abstract

Figures

Contributions To The Literature

Background

Methods

Results

Discussion

Conclusions

Declarations

References

Tables

Supplementary Files

Status:

Version 2