Across virtually every service sector, training and follow-up consultation are cornerstone implementation strategies designed to promote provider uptake and use of evidence-based practices (EBPs; 1). While training and consultation are essential implementation strategies, there is mounting evidence indicating that these strategies alone are insufficient to translate what is known to work in research into routine practices that ultimately improve service recipient outcomes (2, 3). Pre-implementation enhancement strategies (PIES) that complement and increase the yield of training and follow-up consultation are needed, particularly ones that are pragmatic and theoretically informed to target precise mechanisms of behavior change (4, 5). PIES that bolster the yield of implementation strategies is especially needed in the context of youth behavioral health where many youth in need of care do not access quality prevention or treatment services even though numerous EBPs exist (6, 7). The education sector is one of the best settings to develop and test PIES as it is the most common setting where youth access behavioral health services (8, 9, 10). With a triple-blind parallel randomized controlled trial, this study aimed to test a pragmatic PIES designed with principles rooted in social-cognitive theories (SC-PIES) as a complement to EBP training and follow-up consultation to enhance teachers’ intentions to implement and implementation outcomes in the service of producing better youth behavioral health outcomes in the education sector.
Youth Behavioral Health and EBPs in the Education Sector
As a prevalent and impactful youth behavioral health problem, externalizing behavioral disorder and its prevention and treatment remain top priorities for policymakers, researchers, and healthcare professionals in the education sector (11). Externalizing behavioral health problems are outer-directed behaviors that harm relationships with others and violate social norms and rules (12). Youth who exhibit externalizing behavioral problems are likely to experience negative outcomes, including academic difficulties (13), strained relationships with others (14), elevated risk for dropout (15), substance abuse (16), repeated exposure to exclusionary discipline practices and contact with the juvenile justice system (17). To prevent these untoward outcomes, researchers have established several EBPs for use in the educational sector. For example, proactive classroom behavior management (PCBM) includes a suite of EBPs that are prevention-oriented and aim to promote high levels of behavioral engagement in class as incompatible with externalizing behavior problems (18). Extant research has established the effectiveness of various PCBM practices for consistent use by teachers in the context of classrooms where youth spend significant amounts of time (19). PCBM practices promote youth’s behavioral and academic success in class that leads to short-term success in school (e.g., reduced externalizing behaviors, increased academic achievement, and school engagement; 20,21). These short-term outcomes are determinants of longer-term beneficial outcomes that reach into adulthood (22). Although PCBM practices have been disseminated through multiple outlets (e.g., refereed journals, books, and media outlets), passive implementation efforts have proven insufficient to produce scalable changes in teachers’ delivery of PCBM practices (23).
Teachers as Primary Implementers with Varied Response to Training and Consultation
In the education sector, the implementation of universal, prevention-oriented EBPs targeting youth behavioral health outcomes are primarily the responsibility of classroom teachers who spend the most time during the school days interfacing with youth (5). For this reason, teachers are the recipients of training and consultation, which are the most common implementation strategies used to promote teachers’ uptake and delivery of universal EBPs (3). While training and follow-up consultation are considered core implementation strategies, these strategies alone are insufficient to produce quality and consistent implementation across implementers. For example, didactic trainings are necessary to increase knowledge of practices, but they are ineffective alone in producing changes in teachers’ classroom practices (24). Moreover, teachers may demonstrate resistance or ambivalence to change when receiving follow-up support such as consultation (5).
As primary implementers, teachers' motivation plays an instrumental role in determining whether they will respond to training and follow-up consultation to initially adopt then persist towards using an EBP with fidelity (25). Indeed, teachers are a heterogeneous group of implementers who vary significantly according to motivational factors that influence whether they are likely to adopt new practices as part of their regular classroom routines (26,27). Variability in motivational determinants of change exists regardless of whether teachers work in school settings with optimal organizational determinants in place (e.g., supportive leadership, quality training, and coaching; 28). Therefore, there is a need for complementary pre-implementation strategies that occur before training and consultation to target motivational determinants of behavior change to enhance teachers’ responsiveness to training and consultation (29).
Pre-Implementation Enhancement Strategies to Improve Responsiveness to Training and Consultation
Theory-informed and pragmatic pre-implementation enhancement strategies (PIES) offer promise as complementary strategies that can increase the yield of training and follow-up consultation on both implementation and youth behavioral health outcomes. PIES occur before more active implementation supports such as training and consultation. EBP training and consultation more directly focus on increasing providers' knowledge of and follow through with specific practices. A common implicit assumption of training and consultation is that providers are already motivated to change their behaviors. However, this is not always the case. PIES occur before providers participate in training and consultation activities and are designed to motivate and prime providers to engage and respond more to active implementation strategies. Essentially, PIES sits at the immediate intersection of the preparation and active implementation phases of an implementation process (30).
Consistent with recommendations for developing implementation strategies (31), the development of PIES should be grounded in theory to ensure the content targets precise motivational mechanisms of behavior changes before receipt of training and consultation. Theoretically, effective PIES can enhance providers’ motivational mechanisms (e.g., intention to implement), which in turn improve their responsiveness to EBP-specific implementation strategies (e.g., training and consultation) that lead to enhanced implementation outcomes (e.g., adoption, fidelity) that ultimately optimize youth behavioral outcomes.
Use of Social-Cognitive Theory to Develop and Test PIES
Applied social-cognitive theory and research offer principles and evidence that could inform the development of pragmatic and effective PIES (32). In the context of implementation, strategies informed by social-cognitive theory could serve to alter teachers' perceptions and motivation regarding the uptake and use of EBPs, in the context of actively acquiring knowledge about them (training) and participating in ongoing support and feedback to deliver them with fidelity (consultation). Social-cognitive research suggests that implementers’ behavioral intentions are malleable mechanisms of behavior change that can be targeted via intervention (33). The Theory of Planned Behavior (TPB; 34,35) is a widely established social-cognitive theory that has been used to predict and target behavior change (36). The central tenet of the TPB is that one of the best predictors of behavior is a person’s behavioral intentions. Behavioral intentions "capture the motivational factors that influence a behavior; they are indicators of how hard people are willing to try, of how much effort they are planning to exert, in order to perform the behavior” (34). Previous research using the TPB has shown that implementation strategies targeting implementers' behavioral intentions are linked to improved implementation outcomes during the active implementation phase when EBP uptake and use are critical (e.g., 37-39). Moreover, a study conducted with teachers serving youth with autism indicated that, following in-service training, teachers endorsing high intentions to implement were five times more likely to adopt and deliver EBPs than teachers endorsing low intentions to implement (40). As such, implementation strategies that target behavioral intentions before initiating active implementation may serve as an effective approach to increase teachers’ responsiveness to training and consultation.
Several empirically established social-cognitive principles can be used to guide the content development of a PIES (39,35). We strategically selected three social-cognitive principles based on two criteria: (1) empirical evidence related to behavior change and (2) ease of designing content and activities that are perceived as acceptable and pragmatic by teachers. First, the principle of growth mindset was selected as it is a popularized construct that has gained widespread attention in education and can be leveraged for purposes of face validity to engage teachers (41). Research has shown that people’s mindsets or implicit theories about the malleability of their cognitive and physical abilities have significant impacts on their goals, effort, and performance (42).
Second, the saying-is-believing principle was selected, because it involves individuals advocating for an idea or action to others to increase their own commitment to the idea or action regardless of their initial beliefs about it (43). For instance, college youth randomly assigned to write letters to incoming youth endorsing the importance of overcoming social and academic adversity and using problem-solving strategies to do so showed significant improvements in their own school-related behaviors and achievement (44). Teachers who are provided with opportunities to advocate ideas relevant to adopting and implementing new practices may in turn exhibit greater intentions to implement and are more likely to implement once they receive training and follow-up consultative support (45).
Last, we selected the principle of commitment and consistency (46), which involves evoking a state of psychological and emotional tension that gets activated when individuals recognize a discrepancy between a belief and their behavior. The tension and discrepancy lead to an increased likelihood of individuals striving to maintain consistency between their beliefs and actions (46). Specific desired behaviors (i.e., intervention fidelity) can be increased by evoking commitments that are active, public, and voluntary (47). Once individuals make a commitment, they are more motivated to maintain consistency between their beliefs and actions (i.e., commitment or follow through with the stated action). Research has leveraged this principle to induce cognitive dissonance to promote behavior change (48). This technique has been applied to a wide range of behaviors, including voting behavior, fund-raising, and recycling (49,50).
Study Aims
There is limited research on the development and evaluation of PIES as complementary strategies that aim to increase the yield of training and consultation on both implementation and youth behavioral health outcomes. Moreover, the literature on implementation strategies from other service sectors has largely neglected the specification of change mechanisms by which strategies impact implementation and client outcomes (51). A focus on theory-informed mechanisms as mediators of behavior change is vital to identifying why and how implementation strategies (e.g., PIES) work, which can subsequently inform the development, testing, and refinement of effective implementation strategies (52,35). Such inquiries must be informed by valid theories to explain how strategies at different implementation stages (e.g., pre- vs active implementation) influence implementation and youth behavioral health outcomes (53). In the current study, we experimentally examined the efficacy of a pragmatic PIES based on social-cognitive theories (SC-PIES) to influence a putative mechanism of behavior change (intentions to implement), implementation outcomes, and an important youth behavioral health outcome (academic engagement as an incompatible behavior to externalizing disorders). Specifically, we conducted a preliminary triple-blind parallel randomized controlled trial to address four research questions:
(1) As compared to control, does SC-PIES significantly improve teachers’ implementation intentions to implement PCBM practices at posttest after adjusting baseline and covariates?
(2) As compared to control, is receiving SC-PIES associated with significantly improved teachers’ intervention fidelity and class-wide youth behavioral health outcome at the 6-week follow-up after adjusting for baseline and covariates?
(3) Does teachers' intention to implement mediate the association between study condition and intervention fidelity?
(4) Does teachers' intervention fidelity mediate the association between their intentions to implement and class-wide youth behavioral health outcome?