A systematic review of school health policy measurement tools: Implementation determinants and outcomes

doi:10.21203/rs.3.rs-206237/v2

Download PDF

Research

A systematic review of school health policy measurement tools: Implementation determinants and outcomes

https://doi.org/10.21203/rs.3.rs-206237/v2

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background

Governments in some countries or states/provinces mandate school-based policies intended to improve the health and well-being of primary and secondary students, and in some cases the health of school staff. Examples include mandating a minimum time spent per week in programmed physical activity, mandating provision of healthy foods and limiting fat content of school meals, and banning tobacco products or use on school campuses. Although school health researchers have studied whether schools, districts, or states/provinces are meeting requirements, it is unclear to what extent implementation processes and determinants are assessed. The purposes of the present systematic review of quantitative measures of school policy implementation are to identify quantitative measures of implementation determinants and Proctor model implementation outcomes and assess pragmatic and psychometric properties of such measures.

Methods

Peer-reviewed journal articles published 1995–2020 were included if they: 1) had multiple-item quantitative measures of school policy implementation; and 2) addressed overall wellness, tobacco, physical activity, nutrition, obesity prevention, and mental health/bullying/social-emotional learning. The final sample comprised 86 measurement tools from 67 peer-review articles. We extracted study characteristics, such as psychometric and pragmatic measure properties, from included articles based on three frameworks: Implementation Outcomes Framework, 2) Consolidated Framework for Implementation Research, and 3) Policy Implementation Determinants Framework.

Results

Most implementation tools were developed to measure overall wellness policies which combined multiple policy topics (n = 35, 40%) and were in survey form (n = 75, 87%). Fidelity was the most frequently prevalent implementation outcome (n = 70, 81%), followed by adoption (n = 32, 81%). The implementation determinants most assessed were readiness for implementation, including resources (n = 43, 50%), leadership (n = 42, 49%), and policy communication (n = 41, 48%). Overall, measures were low-cost and had easy readability. However, lengthy tools and lack of reported validity/reliability data indicate low transferability.

Conclusions

Implementation science can contribute to more complete and rigorous assessment of school health policy implementation processes, which can improve implementation strategies and ultimately the intended health benefits. Several high-quality measures of implementation determinants and implementation outcomes can be applied to school health policy implementation assessment. D&I researchers can also benefit from measurement experiences of school health researchers.

Health Economics & Outcomes Research

Health Policy

dissemination and implementation

health promotion

measurement

policy

schools

This systematic review provides an innovative summary compilation of identified quantitative measures of school health policy implementation determinants and outcomes.

Dissemination and implementation (D&I) and school health researchers can benefit from sharing expertise to build an integrated understanding of policy implementation. School health researchers are more familiar with these contexts and can guide contextual assessment; whereas D&I researchers can help guide selection of pre-existing measures and pilot testing of adapted assessment tools.

Several high-quality measurement tools tested and used in D&I research can be applied in the school setting to inform policy implementation strategies to improve implementation outcomes and ultimately the intended health benefits.

This review also highlights the need for a focus on health equity as an implementation process and outcome for future study as a means to bridge the gap between policy and practice.

Health policies enacted across multiple levels (e.g., schools, districts, states, countries) are necessary to influence children’s health behaviors (1–6). Although evidence supports the impact of successful school-based policy implementation on student health outcomes (1, 6–10), the disconnect between evidence-based policy and school-based enactment poses challenges for school administrators and teachers (11). Therefore, researchers and practitioners have called for enhanced policy implementation research which specifically targets the implementation determinants, processes, and outcomes, in order to enhance the rate at which polices are adopted and infused in to organizational culture (11–14).

One prominent example of school policy is the Child Nutrition and WIC Reauthorization Act (15), which mandated that all schools participating in the National School Lunch Program (NSLP) within the United States develop a comprehensive wellness policy and a plan for implementation. Another example is the Australian New South Wales (NSW) Sport and Physical Activity Policy, which mandates that all children attending primary and secondary schools should participate in a minimum of 150 minutes of planned moderate activity across the school week (16). Through dissemination and implementation science (D&I) principles, several factors influence the adoption and implementation - herein termed implementation determinants - of school health/physical activity policies and interventions. Such determinants stem from within (e.g., organizational culture, leadership) and outside (e.g., outside collaborations) the school setting, in addition to policy characteristics (e.g., complexity, cost), facilitating or limiting the adoption and diffusion of policy and innovation into practice (17–20). Factors from prior policy/school health promotion implementation research such as lack of funding, training/professional development, and administration support are highlighted as key barriers/negative determinants to implementing health-promotion policies and programs (21–24), whereas provision of such supports are found to be enabling determinants (21). However, the measurement tools used to assess implementation outcomes and determinants remain poorly understood (25), thus contributing to the sustained research-practice gap. The overuse and over-dependence of “barriers” and “facilitators” to explain implementation of school health promotion and policy research can contribute to misinformation and to the circulation of highly-cited factors (i.e., time, funding, support) (17, 22, 26, 27). As such, minimal solutions are provided for stakeholders to better implement policies and programs. Furthermore, much of the earlier research has been conducted through qualitative evaluation (1, 7, 28–31), which offers rich information about implementation processes but limits our ability for generalizability and replication.

Within the field of D&I, a clear distinction exists between implementation outcomes and determinants. Implementation outcomes refer to detectable changes in processes and practices as a result of a particular policy or innovation whereas determinants are factors which can be leveraged to increase the likelihood of implementation success (32–35). Grounded in the model by Proctor et al., implementation outcomes transcend beyond simply “meeting requirements” and include: adoption, acceptability, appropriateness, cost, fidelity, feasibility, penetration, and sustainability (32). Through measuring multiple implementation outcomes, researchers and practitioners are able to see the implementation processes in addition to examining fidelity, which can greatly enhance understanding of how school policies are diffused into practice (32, 36). Further, although research has examined influential factors on school health-related policy implementation, matching these factors to address specific implementation determinants derived through D&I research frameworks (37–39) will allow for greater use in other school health-related policy topics, and increase the credibility of school-based D&I research and practice.

Accordingly, the aims of this systematic review were to: (1) identify quantitative school health policy measurement tools developed to measure implementation at the school, district, or state/provincial levels, (2) describe the policy implementation outcomes and determinants assessed and identify the trends in measurement, and (3) assess pragmatic and psychometric properties of identified implementation measures to understand their quality and suitability for broader application.

This review of school-based policy implementation measures was conducted with a similar protocol from a broad review of health policy implementation tools (36). Both reviews followed procedures conducting a systematic review of implementation measurement tools (40), and adhered to PRISMA reporting guidelines (see Figure 1 and Supplemental Table S1) (41). The review was guided by three prominent D&I frameworks: the Implementation Outcomes Framework (IOF) by Proctor and colleagues (32); the Consolidated Framework for Implementation Research (CFIR) by Damschroder and colleagues (37); and the Policy Implementation Determinants Framework by Bullock and Davis (42, 43). Through a combination of constructs from these frameworks, we sought to gain a deeper understanding of the implementation outcomes, determinants, and processes for school health policy implementation which are assessed through measurement tools. The same definitions of public policy and policy implementation were utilized in accordance with the review by Allen et al. (36). Specifically, public policy includes federal/nation, state/province/county, regional unit, or local level legislation or policies mandated by governmental agencies (44, 45). The implementation of policy conceptualizes the processes by which the mandate is carried out by public or private organizations (45). For the purpose of this review, the organizations of interest comprised states/provinces, school districts, and primary and secondary pre-university schools as implementing sites.

Searches

We searched six databases in April 2019 and again in August 2020 to ensure inclusion of recent articles in the present review: MEDLINE, PsycInfo, and CINAHL Plus through EBSCO; and PAIS, Worldwide Political, and ERIC through ProQuest. We searched terms at four domains: health, public policy, implementation, and measurement; see Supplemental Table S2 for search terms and syntax. Development of the search strings and terms was based on frameworks in D&I and policy research, with details previously described (36).

Inclusion and exclusion criteria

The inclusion criteria comprised English-language peer-reviewed journal articles published from January 1995 through August 2020 and utilized quantitative self-report, observational, and/or archival tools to assess implementation of a government-mandated policy (36). The broad review conducted in 2019 included empiric studies from any continent on policy implementation in any clinical or non-clinical setting on a broad range of health policy topics. Exclusion criteria can be found in Supplementary Table S3. Specific deviations from inclusion/exclusion criteria in the Allen et al. article were: 1) research must have taken place in/with school settings serving students in primary and secondary (ages 5-18; pre-university) schools; 2) measured implementation of school policies already passed or approved that addressed overall wellness, tobacco, physical activity, nutrition, obesity prevention, and mental health/bullying/social-emotional learning; and 3) policy-specific and setting-specific measures were included in the present review but excluded in the initial broad review (which sought generalizable measures that could be applied across multiple settings and topics). Our review included multi-item measures; articles were excluded if the tool included only one relevant item.

Screening

Two members of the research team used Covidence systematic review software (46) to independently screen all abstracts for inclusion and exclusion. Full texts of all empiric studies of school setting public policy implementation that passed abstract screening in 2019 were rescreened independently in summer 2020 by two coauthors (GMM, PA) for potential inclusion into the present review, with decisions and exclusion reasons coded in Excel. The school-setting full text rescreening was conducted because the initial review had excluded measures worded specifically for a certain setting or policy topic, whereas such specific measures were included in the present review. The two coauthors also conducted dual independent full text screening of newly identified 2019-2020 studies that passed abstract screening after the August 2020 updated database searches. The two coauthors met to reach consensus on any inclusion/exclusion disagreements. A third coauthor was consulted if consensus could not be reached. One of the pre-identified exclusion reasons was attributed to each excluded article (for more information see PRISMA chart; Figure 1).

Extraction

A comprehensive extraction procedure was implemented in which co-author (GM, PA, CWB) pairs conducted dual non-independent extraction. A primary reviewer entered relevant information into the extraction database and the secondary reviewer checked data entry for accuracy and completeness. The co-authors met regularly to reach consensus. Information extracted on the measurement properties included: 1) type of measurement tool (i.e., survey, archival, observation); 2) implementation setting (i.e., elementary/primary, middle, high/secondary school, combination of two or more levels); 3) school policy topic (i.e., wellness, physical activity, nutrition, mental health, tobacco, sun safety, etc.); and 4) level of educational entity directing implementation of the governmental mandate (i.e., school, district, state/province, national). Following the three chosen D&I frameworks, all implementation outcomes from the Proctor framework were extracted from measures, followed by selected CFIR constructs which were used in the previous review article and found to be pertinent to policy implementation, and the actor relations/networks and actor context domains from the Bullock and Davis framework. Finally, following the procedures outlined by Lewis and colleagues regarding the Psychometric and Pragmatic Evidence Rating Scale (PAPERS) (40, 47-51), pragmatic (i.e., brevity, cost, readability, training, interpretation) and psychometric (i.e., internal consistency, validity, norms) properties were extracted from each measure to ascertain the quality of each tool. These scoring classifications assign scores from -1 to 4 based on the degree to which the measures meet each criterion; higher scores on each construct reflect higher quality of the measurement tool (Supplemental Tables S4, S5).

Data synthesis

Upon achieving consensus on all measures, descriptive analyses were run to gather frequency of items in each school health policy topic. A subset of tools were widely used and/or based on national samples: the Centers for Disease Control and Prevention School Health Policies and Practices Study (school, district, state) (52); the Wellness School Assessment Tool (53); the Maryland Wellness Policies and Practices Project surveys (school and district level) (54); and the Health Enhancing Physical Activity Europe policy audit (55). We term these “large-scale” tools. Other less frequently reported measures with smaller sample sizes were called “unique tools.” Where appropriate, these measures were analyzed and presented separately when reporting characteristics, given the distinctive differences in methodology and utilization.

Figure 1 shows the PRISMA flowchart which outlines the steps taken from identifying records through database searching to the studies included in the final review. There were 67 studies included in this review; from these 86 measures were extracted for tool characteristics. Of the measures, the vast majority were developed in the United States (n = 60; 69%), followed by Canada (n = 10; 11.6%), European countries (n = 6; 6.9%), and Australia (n = 5; 5.8%). Finally, 2 were developed in India, and 1 each was developed in Indonesia, Mexico, and Taiwan. The 6 studies conducted in Europe were from Denmark (1), the Netherlands (1), Spain (1), or were conducted in multiple countries (3). Table 1 shows the breakdown of tools by school health policy topic and type of tool (i.e., survey, observation, archival). The majority of tools were surveys (n = 75; 87.2%); the most common topic was general wellness policy (i.e., more than two health policy areas; n = 35, 40.6%), followed by nutrition (n = 21; 24.4%) and physical activity (n = 11; 12.7%). Roughly half (n = 42; 49%) of the tool items were generated by experts and 29 measures (33.7%) were piloted with a representative sample. In the included studies, authors reported reliability/validity testing data on pilot testing for 15 measures (17.4%). Of the measures we extracted, psychometric data were available for 28 tools (32.5%).

Table 1

Measures by Policy Topic and Type (N = 86)
	Type of Measurement Tool
Wellness topic	Archival	Observation	Survey
Health Education			2
Mental Health			1
Nutrition	1	1	19
Nutrition and Physical activity			4
Physical Activity	2		9
Sun Safety			3
Tobacco/Drug			9
Wellness Policy	6	1	28
Total	9	2	75

Table 2 displays the implementation outcomes, processes, and determinants extracted for the overall sample and then shown separately for large-scale tools and unique tools. The most common implementation outcomes assessed were Fidelity (n = 70; 81.4%), Adoption (n = 32; 37.2%), and Acceptability (n = 18; 20.9%). The most prevalent implementation determinants in the sample were Actor Relations/Networks (n = 45; 52.3%), followed by Readiness for Implementation- Non-training resources (n = 43; 50.0%) and Leadership for Implementation (n = 42; 48.8%). Figure 2 illustrates the most 10 commonly measured constructs for the whole sample. Tools varied in their assessment of fidelity, ranging from asking respondents to report their implementation on a Likert scale, to asking about implementation of multiple “best practices” and reporting frequency of utilization/execution. Adoption typically manifested through asking respondents about their intentions to implement a policy or practice, or about planning activity which has occurred in order for implementation to be successful.

Table 2

Implementation outcomes and determinants assessed in measurement tools (N = 86), then split by large-scale and unique tools
Domain		Included Measures (N = 86)	(%)	Large-Scale Tools (n = 23)	(%)	Unique Tools (n = 63)	(%)	Definition	Source
Implementation Outcomes	Acceptability	18	20.9%	0	0.0%	18	28.6%	Perceptions by staff in organizations mandated to implement the policy, or perceptions of other stakeholders, that the policy mandate is agreeable, palatable, or satisfactory	Proctor 2011(1)
	Adoption	32	37.2%	10	43.5%	22	34.9%	Intention and initial actions of mandated organizations to revise their organizational policies to address policy mandates (Not policy development or passage of bills into law.)	Proctor 2011
	Appropriateness	9	10.5%	0	0.0%	9	14.3%	Perceived fit, relevance, or compatibility of the [policy] for a given practice setting, provider, or consumer; and/or perceived fit of the [policy] to address a particular issue or problem; context fit	Proctor 2011
	Feasibility	8	9.3%	1	4.3%	7	11.1%	Extent to which a new [policy] can be successfully used or carried out within a given agency or setting; Level of administration required to implement a policy, often called policy automaticity	Proctor 2011
	Fidelity/ Compliance	70	81.4%	21	91.3%	49	77.8%	Degree to which a [policy] was implemented as it was prescribed	Proctor 2011
	Penetration	15	17.4%	8	34.8%	7	11.1%	Integration of a [policy] within a service setting and its subsystems	Proctor 2011
	Sustainability	3	3.5%	1	4.3%	2	3.2%	Extent [new policy] is maintained or institutionalized within a service setting’s ongoing, stable operations	Proctor 2011
	Cost of Implementation	5	5.8%	0	0.0%	5	7.9%	Cost impact of an implementation effort	Proctor 2011
Policy/Innovation Characteristics	Adaptability	3	3.5%	0	0.0%	3	4.8%	Degree to which an intervention can be adapted, tailored, refined, or reinvented to meet local needs	Damschroder, 2009 (2)
	Complexity	3	3.5%	0	0.0%	3	4.8%	Perceived difficulty of implementation, reflected by duration, scope, radicalness, disruptiveness, centrality, and intricacy and number of steps required to implement	Damschroder, 2009
Organizational Characteristics/ Inner Setting	Champions	6	7.0%	0	0.0%	6	9.5%	Field or practice leaders, people who can facilitate and support practice change among professionals	Damschroder, 2009
	Organizational Culture and Climate	9	10.5%	1	4.3%	8	12.7%	Culture: “Norms, values, and basic assumptions of a given organization”; or Climate: “Absorptive capacity for change”, extent policy compliance will be rewarded, supported, and expected within their organization”	Damschroder, 2009; Bullock, 2019 (3)
	Policy Implementation Climate (IC)	4	4.7%	0	0.0%	4	6.3%	Organizational climate specific to the policy mandate	Damschroder, 2009
	IC: Goals & Feedback	6	7.0%	3	13.0%	3	4.8%	Degree [the policy-mandate] goals are clearly communicated, acted upon, and fed back to staff and alignment of that feedback with goals	Damschroder, 2009
	IC: Relative Priority	21	24.4%	2	8.7%	19	30.2%	Individuals’ shared perception of importance of the [policy] implementation within the organization, competing priorities	Damschroder, 2009
	Opinion Leaders	7	8.1%	0	0.0%	7	11.1%	Individuals in an organization who have formal or informal influence on attitudes and beliefs of their colleagues with respect to implementing the policy	Damschroder, 2009
	Readiness to Implement (RI)	5	5.8%	0	0.0%	5	7.9%		Damschroder, 2009
	RI: Communication of policy	41	47.7%	18	78.3%	23	36.5%	Communication plans and channels created for how the regulatory agency or implementing organization/s will disseminate policy mandate content information to implementers. Actions taken to disseminate policy requirements and guidelines to implementers.	Damschroder, 2009
	RI: Policy awareness/ knowledge	27	31.4%	2	8.7%	25	39.7%	Implementing staff/provider awareness the policy mandate exists, or knowledge of policy content	Damschroder, 2009
	RI: Leadership for Implementation	42	48.8%	22	95.7%	20	31.7%	Commitment, involvement, and accountability of leaders and managers with the implementation	Damschroder, 2009
	RI: Non-training Resources	43	50.0%	15	65.2%	28	44.4%	Level of resources dedicated for implementation and on-going operations including money…physical space, and time other than training resources	Damschroder, 2009
	RI: Training	35	40.7%	16	69.6%	19	30.2%	Training of staff/providers in implementing organizations on how to implement the policy-mandated practices	Damschroder, 2009
	Structure of Organization	2	2.3%	0	0.0%	2	3.2%	The social architecture, age, maturity, and size of an organization	Damschroder, 2009
Implementation Process	Enforcement	10	11.6%	1	4.3%	9	14.3%	Strategies used to hold individuals accountable for implementation fidelity/compliance	From screening/ coding
	Evaluation	35	40.7%	18	78.3%	17	27.0%	Quantitative and qualitative feedback about the progress and quality of implementation accompanied with regular personal and team debriefing about progress and experience.	Damschroder, 2009
	General Barriers and Facilitators	20	23.3%	2	8.7%	18	28.6%	Factors which facilitate/enable and hinder implementation	From screening/ coding
	Collaboration	11	12.8%	7	30.4%	4	6.3%	Active involvement of other stakeholders in the organization to implement the policy	From screening/ coding
	Innovation Participants	19	22.1%	10	43.5%	9	14.3%	Engaging individuals who will directly benefit/receive the policy action	Damschroder, 2009
Actor Relationships/ Networks	Actor Relationships/ Networks	45	52.3%	22	95.7%	23	36.5%	Presence and characteristics of relationships between parallel organizations that must collaborate for policy implementation to be effective	Bullock, 2019
	Visibility of policy role & policy actors	23	26.7%	8	34.8%	15	23.8%	Perceived presence and importance of different actors pertinent to implementation of the policy	Bullock, 2019
Actor Context	Political will for policy implementation	12	14.0%	3	13.0%	9	14.3%	Societal desire and commitment to generate resources to carry out policies	Bullock, 2019
	Target population characteristics	1	1.2%	0	0.0%	1	1.6%	Demographics, norms, and neighborhood environments of the population groups that are affecting policy implementation	Bullock, 2019
Other Domain (Not in Manual)	CFIR Process- Planning	2	2.3%	0	0.0%	2	3.2%	The degree to which a scheme or method of behavior and tasks for implementing an intervention are developed in advance, and the quality of those schemes or methods.	Damschroder, 2009
	CFIR Innovation Characteristics- Relative advantage	1	1.2%	0	0.0%	1	1.6%	Stakeholders’ perception of the advantage of implementing the intervention versus an alternative solution.	Damschroder, 2009
	CFIR Inner Setting- Tension for change	1	1.2%	0	0.0%	1	1.6%	The degree to which stakeholders perceive the current situation as intolerable or needing change.	Damschroder, 2009
1. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda. Administration and Policy in Mental Health and Mental Health Services Research. 2011;38(2):65–76.
2. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Science. 2009;4(1):50-.
3. Bullock HL. Understanding the implementation of evidence-informed policies and practices from a policy perspective: a critical interpretive synthesis in: How do systems achieve their goals? the role of implementation in mental health systems improvement [Dissertation]. Hamilton, Ontario: McMaster University; 2019.

For large-scale tools, the most commonly measured determinants were the CFIR readiness for implementation- leadership construct and actor relationships/networks (both n = 22; 95.7%), and the most commonly measured outcome was fidelity from the Proctor model (n = 21; 91.3%). Compared to the whole sample, some construct which were prevalent in large-scale tools only were the outcome of penetration and the innovation participants determinant from CFIR- implementation process construct (both n = 8; 34.7%). Among unique tools, fidelity was also the most commonly measured outcome (n = 49; 77.8%) with readiness for implementation- non-training resources (n = 28; 44.4%) as the most common determinant. In terms of least measured constructs, target population characteristics affecting implementation (n = 1) and structure of organization from the CFIR inner setting domain (n = 2) were least measured in the entire sample (see Table 2 for all constructs).

The PAPERS pragmatic scores are shown in Fig. 3 and show separate median scores for the large-scale and unique tools. In terms of brevity, large-scale tools were scored lower as they had a greater number of items (average = 150) compared to unique tools (average = 73). Almost all tools were free or available at very minimal cost to the public (i.e., not required to pay for article and tool if not subscribed to journal), although our team needed to request original items from the corresponding authors for a large proportion of the sample. Large scale tools scored higher on training for tool administration as most required no/minimal training, compared to unique tools which were often described more ambiguously. However, the unique tools were shorter, provided easier interpretation guidelines, and had lower grade-level reading scores than the larger-scale tools.

Psychometric PAPERS scores were low (0 median) across all components, with large-scale tools generally demonstrating higher quality according to internal consistency and validity (0.66 versus 0.56 mean PAPERS score, out of a possible lowest score of -1 to a possible highest score of 4). Overall, internal consistency α coefficient scores ranged from 0.40 to 0.98 across the studies. In addition, the sample sizes (2 versus 0.78 mean PAPERS score) used to deduce findings were larger for large-scale studies, ranging from 19 (56) to 6,504 schools (57); samples ranged in between these numbers and were at the student, teacher, school, district, and state/provincial level (See Supplemental Tables S4, S5 for scoring criteria). Very few tool development articles/documents provided concurrent and structural validity information; none of the large-scale tool studies provided such information. Overall, psychometric quality of tools was unknown or low. These results highlight areas for improvement in future tool development and reporting.

Characteristics and PAPERS scores for each tool are provided in Supplementary Table S6. Despite low scores overall, some tools were well-developed and validated according to best practices. One example is the Maryland Wellness Policies and Practices Project (MWPPP) district and school surveys (54), which each received a score of at least 10 (15 for district, 10 for school) for pragmatic and 12 for psychometric properties (Supplementary Table S6). This tool measures overall wellness policy implementation at the school and district levels, assessing multiple implementation outcomes (i.e., adoption, feasibility, fidelity) and determinants (i.e., implementation climate- goals and feedback; readiness – communication of policy, policy awareness/knowledge, leadership, non-training resources, training; actor relationships, visibility of policy role, evaluation, collaboration, innovation participants). This tool may be easily adapted for use within other states and countries depending on policy characteristics. Supplemental Table S6 displays PAPERS scores and tool characteristics for all 86 measured included in the review.

The purpose of this study was to obtain a comprehensive understanding of quantitative implementation measurement tools for school health policy following a systematic review protocol. Findings revealed a large number of tools which covered a wide range of policy topics and implementation settings, with general wellness policy as the most commonly measured area of health promotion. Most of the tools assessing wellness policies more broadly were from the United States, which aligns with federal mandates for schools to develop and implement comprehensive wellness programming (15). Further, it should be noted that almost all tools were gathered from high-income countries, which draws attention to low- and middle-income countries (LMIC) and the potential for both policy development and implementation evaluation as a means to support ongoing health needs in such populations. Findings from systematic review research highlight a lack of policy/intervention initiatives from LMIC which sought to address child health promotion in the school setting (58). Accordingly, further work is warranted to examine the fit of existing tools for school contexts in LMIC countries, and to determine how tools from high-income countries may be adapted for use in LMIC to optimize efficiency and sharing of resources.

Integration of three prominent implementation frameworks in this study facilitated a rich understanding of implementation processes, outcomes, and determinants in a policy context. The finding that fidelity was the most commonly assessed implementation outcome aligns with findings of the broader review by Allen et al. (36), and highlights the high dependence on fidelity as an indicator of implementation success. Several tools only measured fidelity and/or adoption as the implementation outcomes, which draws concern for addressing constructs such as feasibility and sustainability, among others. Only 8 measures addressed feasibility, with 7 of those within unique tools; this is somewhat contradictory to the extant literature on school-based programming, as many studies have reported low feasibility for implementing policies and health promoting interventions (26, 59–61). Further, through qualitative and mixed-methods research it has transpired that, despite providing financial and logistical support to schools, districts, and states/provinces, most policies are difficult to sustain in absence of such support (61–63). The finding that only 3 tools measured sustainability is concerning given the emphasis on sustainability/maintenance as a key weakness in implementation science and policy research (64, 65). Accordingly, it is clear that a greater emphasis on other implementation outcomes and processes would be beneficial in school policy research, given the top-down nature of policy to practice and need to understand how policy and practice can be sustained over time. Measures of implementation outcomes are continuously being developed and tested for validity and reliability. For example, brief measures of acceptability, feasibility, and appropriateness were designed to add in a specific evidence-based practice (or policy) as the item referent; these have preliminary evidence for good reliability and validity (66). Luke and colleagues developed a measure to assess organizational capacity for sustaining public health and other programs that is reliable and has been tested for construct validity (67). Although many established measures in the implementation science literature have been developed for use in clinical or non-school based community settings, there may be potential for use in schools.

The finding that readiness for implementation as a general construct was most measured reflects prior research stressing the importance of assessing readiness and organizational capacity for implementation (22, 68–73). Within this broader construct, non-training resources was the most common determinant assessed; provision of financial resources and personnel support have been cited as supportive factors for policy and innovation implementation in school research (9, 74, 75). Following this, leadership for implementation was very prevalent in the measures, which again reflects extant knowledge that new innovations require a leader to succeed (61, 76, 77). Finally, the prevalence of items measuring communication of policy demonstrated the importance of engaging stakeholders in policy implementation through enhancing awareness of such initiatives. Such communication is somewhat understudied as a determinant of implementation in school-based literature based on systematic review research (78, 79), but is perhaps one of the most influential determinants of implementation success. School policy research may be further enhanced by studying the relationships between implementation determinants and outcomes to provide clearer evidence between frameworks such as CFIR and the Proctor outcomes framework (37, 65). Further, rather than developing completely new measurement tools, those previously tested in community and clinical settings may be used as is or adapted for school settings, facilitating transferability through implementation science (25).

Unlike readiness for implementation, there were a lack of measures to assess the inner setting and implementation process domains, with relative priority (inner setting) and evaluation (implementation process) identified as the most common among constructs. Research has demonstrated the importance of studying organizational culture and climate as a determinant of implementation (76), given that teachers’ actions are encompassed by school- and district-level policies and practices (80, 81). Some innovations have indeed failed despite leadership for implementation (i.e., small group of leaders taking ownership) due to conflicting organizational practices and lack of priority placed on such initiatives (24, 82). For true diffusion of innovation to occur, institutional buy-in is essential (83, 84); future measures development should therefore integrate these constructs as a means to better understand what impacts policy implementation and bridge the research to practice gap.

Application of the PAPERS rating criteria for pragmatic and psychometric properties revealed areas of strength and need for future improvement (40, 50, 85). Findings for the pragmatic criteria demonstrated that school policy implementation measures found were generally low-cost and written to a lay audience. However, many tools were long and median scores were driven by large-scale tools such as the SHPPS (52, 86); a key barrier to conducting research and evaluation with schools is the limited time that stakeholders are able to spend completing surveys and other audit tools, which has implications for data quality and reliability (30, 85, 87, 88). Although a key need from this study is to adopt pre-existing or develop comprehensive measures which examine implementation outcomes, processes, and determinants, this can lead to lengthy measurement tools which can become arduous to complete and lead to disenfranchisement from stakeholders. Finally, psychometric PAPERS protocols revealed that efforts to ensure quality of tools centered mainly on analyzing internal consistency, with little attention paid to other forms of validity and reliability. This trend is common across other reviews of implementation measures (47, 89), and has implications for broader tool use, specifically when trying to demonstrate implementation efficacy to other populations or policies within school settings. Accordingly, careful tool development should be a focus, and over time it may transpire that some determinants are more influential than others in the policy implementation field, facilitating a streamlined process for subsequent evaluation. Best practices such as field-based pilot testing based on representative samples and developing input from experts are therefore essential in enhancing the pragmatic capabilities of these tools.

Although we conducted a rigorous systematic review following previously established protocols, there are several limitations to note. First, we only extracted tools which were available through online library searches and contacting authors directly where we could not find measures online. We used several approaches to retrieve all tools for extraction, but some tools were unavailable online or from the study authors. We were unable to analyze tools for which we could not access original items. Tools from the grey literature were also not included in this review; although we searched for manuals and tools available online, it was required they were cited in a peer-reviewed article first. Second, we did not explicitly screen for health equity constructs, but based on our review of included tools there was not much to be gleaned in terms of health equity and policy. Several implementation science frameworks integrate health equity and these help to provide guidance for future measurement development (90–96). Finally, although we took a comprehensive policy approach, some policy topics were excluded (i.e., not directly related to health/wellness topics), and in excluding these we may have overlooked other pertinent measurement tools.

What gets measured gets achieved (97) – our review suggests that more comprehensive measurement tools are needed for school policy research that come from or could potentially be transferred to other settings (i.e., community, clinical). Enhancing the quality of policy D&I research through high-quality pragmatic measures will mark a key step in bridging the policy to practice gap (48, 98, 99). Furthermore, given the lack of focus on addressing health equity, there is now an opportunity to develop tools which can help distinguish practices that worsen or mitigate health disparities. The WIC Child Nutrition Re-Authorization act (15) and USDA Healthy Hunger Free Kids Act (HHFKA)(100) are examples of policies which inherently are aimed at reducing health inequality given the focus on NSLP integration, but we know little about how their implementation may influence social determinants of health. Thus, more explicitly addressing health equity is a priority for future research and practice in health policy, in order to elicit a meaningful impact on population health.

Ethics approval and consent to participate: Not applicable.

Consent for publication: Not applicable.

Availability of data and materials: A compendium of identified measures is publicly available for dissemination at https://www.health-policy-measures.org/. A link is provided on the website of the Prevention Research Center, Brown School, Washington University in St. Louis, at https://prcstl.wustl.edu/. The authors invite interested organizations to provide a link to the compendium on their own websites. Citations and abstracts of excluded policy-specific measures are available on request.

Competing interests: The authors declare that they have no competing interests.

Funding: Funding support for this project is from the National Cancer Institute P50CA244431, the Centers for Disease Control and Prevention U48DP006395, and the Foundation for Barnes-Jewish Hospital, with support from the Washington University in St. Louis Institute of Clinical and Translational Science Pilot Program, NIH/National Center for Advancing Translational Sciences (NCATS) grant UL1 TR002345.

Authors' contributions: Eligibility criteria: PA, RCB, CWB, GMM. Search strings and terms: PA, CWB. Abstract screening: PA, CWB. Full-text screening: GMM, PA. Data extraction: GMM, CWB, PA. Data aggregation: GMM, PA. Writing: GMM, CWB, PA, RCB. Editing: GMM, CWB, PA, RCB. The authors read and approved the final manuscript.

Acknowledgements: We wish to acknowledge the administrative support of Mary Adams, Linda Dix, and Cheryl Valko at the Prevention Research Center, Brown School, Washington

University in St. Louis. We also wish to thank Mia Vogel for her support in early refinements to data extraction methods. The Washington University library system provided assistance in developing and testing search syntax in this systematic review and facilitated inter-library loans.

Authors' information (optional): GMM and RCB are with the Implementation Science Center for Cancer Control and Prevention Research Center, Brown School, Washington University in St. Louis, and the Division of Public Health Sciences (Department of Surgery), Washington University School of Medicine, Washington University in St. Louis. CWB and PA are with the Prevention Research Center, Brown School, Washington University in St. Louis.

Abbott R, Macdonald D, Hay P, McCuaig L. "Just Add Facilitators and Stir": Stimulating Policy Uptake in Schools. Educational Management Administration Leadership. 2011;39(5):603–20.
Chriqui JF, Chaloupka FJ. Transparency and Oversight in Local Wellness Policies. J Sch Health. 2011;81(2):114–21.
Fagen MC, Asada Y, Welch S, Dombrowski R, Gilmet K, Welter C, et al. Policy, systems, and environmentally oriented school-based obesity prevention: Opportunities and challenges. Journal of Prevention Intervention in the Community. 2014;42(2):95–111.
Schwartz MB, Henderson KE, Falbe J, Novak SA, Wharton CM, Long MW, et al. Strength and Comprehensiveness of District School Wellness Policies Predict Policy Implementation at the School Level. J Sch Health. 2012;82(6):262–7.
Story M, Nanney MS, Schwartz MB. Schools and obesity prevention: Creating school environments and policies to promote healthy eating and physical activity. Milbank Q. 2009;87:71–100.
Turner L, Leider J, Piekarz-Porter E, Schwartz MB, Merlo C, Brener N, et al. State Laws Are Associated with School Lunch Duration and Promotion Practices. Journal of the Academy of Nutrition Dietetics. 2018;118(3):455–63.
Allison KR, Vu-Nguyen K, Ng B, Schoueri-Mychasiw N, Dwyer JJM, Manson H, et al. Evaluation of Daily Physical Activity (DPA) policy implementation in Ontario: surveys of elementary school administrators and teachers. BMC Public Health. 2016;16(1):1–16.
Baker EA, Elliott M, Barnidge E, Estlund A, Brownson RC, Milne A, et al. Implementing and Evaluating Environmental and Policy Interventions for Promoting Physical Activity in Rural Schools. J Sch Health. 2017;87(7):538-.
Cradock AL, Barrett JL, Carter J, McHugh A, Sproul J, Russo ET, et al. Impact of the Boston active school day policy to promote physical activity among children. American Journal of Health Promotion. 2014;28:54–64.
Melard N, Grard A, Robert PO, Kuipers MAG, Schreuders M, Rimpela AH, et al. School tobacco policies and adolescent smoking in six European cities in 2013 and 2016: A school-level longitudinal study. Prev Med. 2020;138:106142.
Turgeon B. A district wellness policy: The gap between policy and practice. Journal of Education Learning. 2013;2(3):26–38.
Brownson RC, Eyler AA, Harris JK, Moore JB, Tabak RG. Getting the Word Out. Journal of Public Health Management Practice. 2018;24(2):102–11.
Harvey SP, Markenson D, Gibson CA. Assessing School Wellness Policies and Identifying Priorities for Action: Results of a Bi-State Evaluation. The Journal Of School Health. 2018;88(5):359–69.
Vandevijvere S, Mackay S, Swinburn B. Measuring and stimulating progress on implementing widely recommended food environment policies: the New Zealand case study. Health Research Policy Systems. 2018;16:1-N. .PAG.
Child Nutrition. and WIC Reauthorization Act of 2004, (2004).
Education NSWDo. Sport and Physical Activity Policy 2020 [Available from: https://policies.education.nsw.gov.au/policy-library/policies/sport-and-physical-activity-policy.
Agron P, Berends V, Ellis K, Gonzalez M. School wellness policies: Perceptions, barriers, and needs among school leaders and wellness advocates. J Sch Health. 2010;80:527–35.
Budd EL, Schwarz C, Yount BW, Haire-Joshu D. Factors influencing the implementation of school wellness policies in the United States, 2009. Prev Chronic Dis. 2012;9:E118.
Lucarelli JF, Alaimo K, Mang E, Martin C, Miles R, Bailey D, et al. Facilitators to promoting health in schools: Is school health climate the key? J Sch Health. 2014;84:133–40.
McDonnell E, Probart C. School Wellness Policies: Employee Participation in the Development Process and Perceptions of the Policies. Journal of Child Nutrition & Management. 2008;32(1).
Alexander KE, Brijnath B, Mazza D. Barriers and enablers to delivery of the Healthy Kids Check: An analysis informed by the Theoretical Domains Framework and COM-B model. Implementation Science. 2014;9(1):60-.
Blaine RE, Franckle RL, Ganter C, Falbe J, Giles C, Criss S, et al. Using school staff members to implement a childhood obesity prevention intervention in low-income school districts: The massachusetts childhood obesity research demonstration (MACORD Project), 2012–2014. Preventing Chronic Disease. 2017.
Bozsik F, Berman M, Shook R, Summar S, DeWit E, Carlson J. Implementation contextual factors related to youth advocacy for healthy eating and active living. Translational Behavioral Medicine. 2018;8(5):696–705.
Slawson D, Southerland J, Lowe E, Dalton W, Pfortmiller D, Schetzina K. Go Slow Whoa meal patterns: Cafeteria staff and teacher perceptions of effectiveness in winning with wellness schools. J Sch Health. 2013;83:485–92.
Martinez RG, Lewis CC, Weiner BJ. Instrumentation issues in implementation science. Implementation Science. 2014;9(1):118.
McLoughlin GM, Graber KC, Woods AM, Templin T, Metzler M, Khan NA. The Status of Physical Education Within a Nationally Recognized School Health and Wellness Program. Journal of Teaching in Physical Education. 2020;39(2):274–83.
Slawson DL, Southerland J, Lowe EF, Dalton WT, Pfortmiller DT, Schetzina K. Go slow whoa meal patterns: Cafeteria staff and teacher perceptions of effectiveness in winning with wellness schools. J Sch Health. 2013;83(7):485–92.
Asada Y, Ziemann M, Zatz LY, Chriqui J. Successes and challenges in school meal reform: Qualitative insights from food service directors. J Sch Health. 2017;87(8):608–15.
Evenson KR, Ballard K, Lee G, Ammerman A. Implementation of a school-based state policy to increase physical activity. J Sch Health. 2009;79:231–8.
Phillips MM, Raczynski JM, West DS, Pulley L, Bursac Z, Leviton LC. The evaluation of Arkansas Act 1220 of 2003 to reduce childhood obesity: Conceptualization, design, and special challenges. Am J Community Psychol. 2013;51(1–2):289–98.
Weatherson KA, McKay R, Gainforth HL, Jung ME. Barriers and facilitators to the implementation of a school-based physical activity policy in Canada: application of the theoretical domains framework. BMC Public Health. 2017;17(1):835.
Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76.
Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health. 2009;36(1):24–34.
Purtle JD, Brownson EA, RC. Policy Dissemination Research. In: Brownson R,; Colditz, GA; Proctor, EK, editor. Dissemination and Implementation Research in Health: Translating Science to Practice, Second Edition. New York: Oxford University Press; 2018.
Tabak RG, Padek MM, Kerner JF, Stange KC, Proctor EK, Dobbins MJ, et al. Dissemination and Implementation Science Training Needs: Insights From Practitioners and Researchers. Am J Prev Med. 2017;52(3S3):322-S9.
Allen P, Pilar M, Walsh-Bailey C, Hooley C, Mazzucca S, Lewis CC, et al. Quantitative measures of health policy implementation determinants and outcomes: a systematic review. Implement Sci. 2020;15(1):47.
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Science. 2009;4(1):50-.
Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A systematic review of the use of the Consolidated Framework for Implementation Research. Implementation Science. 2016;11(1):72-.
Waltz TJ, Powell BJ, Fernández ME, Abadie B, Damschroder LJ. Choosing implementation strategies to address contextual barriers: diversity in recommendations and future directions. Implementation Science. 2019;14(1):42.
Lewis CC, Mettert KD, Dorsey CN, Martinez RG, Weiner BJ, Nolen E, et al. An updated protocol for a systematic review of implementation-related measures. Syst Rev. 2018;7(1):66.
Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097.
Bullock HL. Understanding the implementation of evidence-informed policies and practices from a policy perspective: a critical interpretive synthesis in: How do systems achieve their goals? the role of implementation in mental health systems improvement [Dissertation]. Hamilton, Ontario: McMaster University; 2019.
Bullock HL, Lavis JN. Understanding the supports needed for policy implementation: a comparative analysis of the placement of intermediaries across three mental health systems. Health Research Policy Systems. 2019;17(1):82.
Eyler AACJ, Moreland-Russell S, Brownson RC, editors. Prevention, Policy, and Public Health, First Edition. New York: Oxford University Press; 2016.
Natesan SDM. R.R. Literature review of public policy implementation. Int J Public Policy. 2015;11(4):219–38.
Covidence systematic review software. Melbourne, Australia: Veritas Health Innovation.
Lewis CC, Fischer S, Weiner BJ, Stanick C, Kim M, Martinez RG. Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria. Implement Sci. 2015;10:155.
Glasgow RE, Riley WT. Pragmatic measures: what they are and why we need them. Am J Prev Med. 2013;45(2):237–43.
Henrikson NB, Blasi PR, Dorsey CN, Mettert KD, Nguyen MB, Walsh-Bailey C, et al. Psychometric and Pragmatic Properties of Social Risk Screening Tools: A Systematic Review. Am J Prev Med. 2019;57(6S1):13–24.
Powell BJ, Stanick CF, Halko HM, Dorsey CN, Weiner BJ, Barwick MA, et al. Toward criteria for pragmatic measurement in implementation research and practice: a stakeholder-driven approach using concept mapping. Implement Sci. 2017;12(1):118.
Stanick CF, Halko HM, Nolen EA, Powell BJ, Dorsey CN, Mettert KD, et al. Pragmatic measures for implementation research: development of the Psychometric and Pragmatic Evidence Rating Scale (PAPERS). Translational Behavioral Medicine. 2019.
Cdc. School Health Policies and Practices Study [SHPPS]. 2012.
Rudd Center for Food P. Obesity. WellSAT: Wellness School Assessment Tool.
Hager ER, Rubio DS, Eidel GS, Penniston ES, Lopes M, Saksvig BI, et al. Implementation of Local Wellness Policies in Schools: Role of School Systems, School Health Councils, and Health Disparities. J Sch Health. 2016;86(10):742–50.
Bull FC, Milton K, Kahlmeier S. National policy on physical activity: The development of a policy audit tool. Journal of Physical Activity Health. 2014;11(2):233–40.
Bassi S, Gupta VK, Park M, Nazar GP, Rawal T, Bhaumik S, et al. School policies, built environment and practices for non-communicable disease (NCD) prevention and control in schools of Delhi, India. PLoS One. 2019;14(4):e0215365.
Kehm R, Davey CS, Nanney MS. The role of family and community involvement in the development and implementation of school nutrition and physical activity policy. J Sch Health. 2015;85(2):90–9.
Xu T, Tomokawa S, Gregorio ER Jr, Mannava P, Nagai M, Sobel H. School-based interventions to promote adolescent health: A systematic review in low- and middle-income countries of WHO Western Pacific Region. PloS one. 2020;15(3):e0230046-e.
Au LE, Ritchie LD, Gurzo K, Nhan LA, Woodward-Lopez G, Kao J, et al. Post-Healthy, Hunger-Free Kids Act Adherence to Select School Nutrition Standards by Region and Poverty Level: The Healthy Communities Study. J Nutr Educ Behav. 2020;52(3):249–58.
Gillies C, Alexander Research C, Farmer A, Maximova K, Willows ND. Alexander First Nations Parents' Perceptions of a School Nutrition Policy. Can J Diet Pract Res. 2020;81(2):66–71.
McLoughlin GM, Candal P, Vazou S, Lee JA, Dzewaltowski DA, Rosenkranz RR, et al. Evaluating the implementation of the SWITCH® school wellness intervention and capacity-building process through multiple methods. International Journal of Behavioral Nutrition Physical Activity. 2020;17(1):162.
Rozema AD, Mathijssen JJP, Jansen MWJ, van Oers JAM. Sustainability of outdoor school ground smoking bans at secondary schools: a mixed-method study. European Journal Of Public Health. 2018;28(1):43–9.
Wolfenden L, Reilly K, Kingsland M, Grady A, Williams CM, Nathan N, et al. Identifying opportunities to develop the science of implementation for community-based non-communicable disease prevention: A review of implementation trials. Prev Med. 2019;118:279–85.
Glasgow RE, Dzewaltowski DA, Estabrooks PA, Gaglio BA, King D, Klesges L. RE-AIM. 2010.
Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda. Administration Policy in Mental Health Mental Health Services Research. 2011;38(2):65–76.
Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. Psychometric assessment of three newly developed implementation outcome measures. Implementation science: IS. 2017;12(1):108-.
Luke DA, Calhoun A, Robichaux CB, Elliott MB, Moreland-Russell S. The Program Sustainability Assessment Tool: a new instrument for public health programs. Prev Chronic Dis. 2014;11:130184.
Scaccia JP, Cook BS, Lamont A, Wandersman A, Castellow J, Katz J, et al. A practical implementation science heuristic for organizational readiness: R = MC. Journal of community psychology. 2015;43(4):484-.
Cheung K, Lesesne CA, Rasberry CN, Kroupa E, Fisher D, Robin L, et al. Barriers and facilitators to sustaining school health teams in coordinated school health programs. Health Promotion Practice. 2017;18:418–27.
Holt DT, Helfrich CD, Hall CG, Weiner BJ. Are you ready? How health professionals can comprehensively conceptualize readiness for change. J Gen Intern Med. 2010;25(SUPPL. 1):50–5.
Lee JA, Welk GJ, Vazou S, Ellingson LD, Lanningham-Foster L, Dixon P. Development and application of tools to assess elementary school wellness environments and readiness for wellness change (doctoral dissertation) 2018.
Octaria Y, Apriningsih A, Dwiriani CM, Februhartanty J. School readiness to adopt a school-based adolescent nutrition intervention in urban Indonesia. Public Health Nutr. 2020:1–12.
Weiner BJ. A theory of organizational readiness for change. Implementation Science. 2009;4(1):67-.
Nathan N, Wiggers J, Bauman AE, Rissel C, Searles A, Reeves P, et al. A cluster randomised controlled trial of an intervention to increase the implementation of school physical activity policies and guidelines: study protocol for the physically active children in education (PACE) study. BMC Public Health. 2019;19(1):170.
Nathan NK, Sutherland RL, Hope K, McCarthy NJ, Pettett M, Elton B, et al. Implementation of a School Physical Activity Policy Improves Student Physical Activity Levels: Outcomes of a Cluster-Randomized Controlled Trial. J Phys Act Health. 2020:1–10.
Fernandez ME, Walker TJ, Weiner BJ, Calo WA, Liang S, Risendal B, et al. Developing measures to assess constructs from the Inner Setting domain of the Consolidated Framework for Implementation Research. Implement Sci. 2018;13(1):52.
McIsaac JLD, Read K, Veugelers PJ, Kirk SFL. Culture matters: A case of school health promotion in Canada. Health Promotion International. 2017;32(2).
McKay H, Naylor P-J, Lau E, Gray SM, Wolfenden L, Milat A, et al. Implementation and scale-up of physical activity and behavioural nutrition interventions: an evaluation roadmap. International Journal of Behavioral Nutrition Physical Activity. 2019;16(1):102-.
Naylor P-J, Nettlefold L, Race D, Hoy C, Ashe MC, Wharf Higgins J, et al. Implementation of school based physical activity interventions: A systematic review. Prev Med. 2015;72:95–115.
Langille J-LD, Rodgers WM. Exploring the influence of a social ecological model on school-based physical activity. Health Education Behavior. 2010;37:879–94.
Suarez-Balcazar Y, Redmond L, Kouba J, Hellwig M, Davis R, Martinez L, et al. Introducing systems change in the schools: the case of school luncheons and vending machines. Am J Community Psychol. 2007;39(3):335–45.
McIsaac JD, Penney TL, Masse L, Kirk SFL. The Association between Perceived Adequacy and Capacity for School Food Policy Implementation with Food Availability and Policy Adherence in Nova Scotia, Canada. Int J Environ Res Public Health. 2019;16(11).
Dearing JW, Cox JG. Diffusion of innovations theory, principles, and practice. Health Aff. 2018;37(2):183-.
Rogers EM. Diffusion of Innovations, Fifth Edition2003.
Stanick CF, Halko HM, Nolen EA, Powell BJ, Dorsey CN, Mettert KD, et al. Pragmatic measures for implementation research: development of the Psychometric and Pragmatic Evidence Rating Scale (PAPERS). Transl Behav Med. 2019.
Centers for Disease C, Prevention. SHPPS 2006: School Health Policies and Programs Study–State-Level School Health Policies and Practices. A State-by-State Summary from the School Health Policies and Programs Study 2006. Atlanta, GA: Centers for Disease Control and Prevention; 2007.
Cargo M, Mercer SL. The value and challenges of participatory research: Strengthening its practice. Annu Rev Public Health. 2008;29(1):325–50.
Harrington KF, Binkley D, Reynolds KD, Duvall RC, Copeland JR, Franklin F, et al. Recruitment issues in school-based research: Lessons learned from the High 5 Alabama project. J Sch Health. 1997;67(10):415–21.
Allen P, Pilar M, Walsh-Bailey C, Hooley C, Mazzucca S, Lewis CC, et al. Quantitative measures of health policy implementation determinants and outcomes: a systematic review. Implementation Science. 2020;15(1):47.
Woodward EN, Matthieu MM, Uchendu US, Rogal S, Kirchner JE. The health equity implementation framework: proposal and preliminary study of hepatitis C virus treatment. Implementation Science. 2019;14(1):26-.
Shelton RC, Chambers DA, Glasgow RE. An Extension of RE-AIM to Enhance Sustainability: Addressing Dynamic Context and Promoting Health Equity Over Time. Frontiers in Public Health. 2020;8(134).
Eslava-Schmalbach J, Garzón-Orjuela N, Elias V, Reveiz L, Tran N, Langlois EV. Conceptual framework of equity-focused implementation research for health programs (EquIR). International Journal for Equity in Health. 2019;18(1):80.
Aby MJ. Race and Equity in Statewide Implementation Programs: An Application of the Policy Ecology of Implementation Framework. Adm Policy Ment Health. 2020;47(6):946–60.
Dover DC, Belon AP. The health equity measurement framework: a comprehensive model to measure social inequities in health. International Journal for Equity in Health. 2019;18(1):36.
Yousefi Nooraie R, Kwan BM, Cohn E, AuYoung M, Clarke Roberts M, Adsul P, et al. Advancing health equity through CTSA programs: Opportunities for interaction between health equity, dissemination and implementation, and translational science. J Clin Transl Sci. 2020;4(3):168–75.
Nápoles AM, Stewart AL. Transcreation: an implementation science framework for community-engaged behavioral interventions to reduce health disparities. BMC Health Serv Res. 2018;18(1):710-.
Thacker SB. Public health surveillance and the prevention of injuries in sports: what gets measured gets done. J Athl Train. 2007;42(2):171–2.
Glasgow RE. What Does It Mean to Be Pragmatic? Pragmatic Methods, Measures, and Models to Facilitate Research Translation. Health Education Behavior. 2013;40(3):257–65.
Glasgow RE, Estabrooks PE. Pragmatic Applications of RE-AIM for Health Care Initiatives in Community and Clinical Settings. Prev Chronic Dis. 2018;15:E02.
Healthy. Hunger-Free Kids Act of 2010, (2010).

SupplementalTablesSchoolPolMeasRevFINAL.docx

Download PDF

Review #2 received at journal
13 May, 2021
Review #1 received at journal
18 Apr, 2021
Reviewer #2 agreed at journal
11 Apr, 2021
Reviews received at journal
24 Mar, 2021
Reviewer #1 agreed at journal
24 Mar, 2021
Reviewers invited by journal
20 Mar, 2021
Editor assigned by journal
08 Feb, 2021
Submission checks completed at journal
08 Feb, 2021
Editor invited by journal
08 Feb, 2021
First submitted to journal
08 Feb, 2021

You are reading this latest preprint version

A systematic review of school health policy measurement tools: Implementation determinants and outcomes

Status:

Version 2

Abstract

Background

Methods

Results

Conclusions

Figures

Contributions To The Literature

Introduction

Methods

Results

Discussion

Limitations

Conclusions

Declarations

References

Supplementary Files

Status:

Version 2