Aim
This study is part of a larger effectiveness-implementation trial of the parent training intervention SPARCK. In a randomized controlled trial (RCT), SPARCK’s clinical effectiveness in preventing and reducing negative outcomes and promoting positive outcomes in children and parents is investigated. A separate study protocol is written for the RCT (21). In order to grant appropriate attention and space to the implementation part of the project, a separate implementation study protocol was deemed warranted. In the current study, we aim to investigate the relations between implementation determinants (30, 31) and successful implementation of SPARCK, measured by four key implementation outcomes, namely fidelity, acceptability, appropriateness, and feasibility (5). In addition, we examine how implementation determinants and implementation outcomes are associated with clinical outcomes of the intervention. Finally, implementation strategies used in the trial will be specified and reported (25, 39). The combined RCT and implementation study will add to the intervention and implementation knowledge field by investigating the effectiveness of an intervention tailored to the needs of the client families, as well as to the needs and context of the service providers at multiple levels and multiple sites. Together, this may result in improved clinical effect, better contextual fit, greater implementation success, and reduction of the time lag between research findings and usage in the practice field (18, 43, 44).
The overall aim of the SPARCK project, including effectiveness and implementation, is to innovate municipal level frontline services with a usable and effective parent training intervention for prevention and reduction of childhood mental health problems. We have five research hypotheses to guide the investigation of implementation of SPARCK; 1) higher practitioner fidelity to SPARCK is associated with better clinical outcomes; 2) participants’ positive ratings of facilitators and negative ratings on barriers are associated with better clinical outcomes; 3) participants’ positive ratings of facilitators and negative ratings on barriers are associated with better implementation outcomes, 4) better implementation outcomes are associated with better clinical outcomes, and 5) participants from municipalities that engaged in co-creation of SPARCK will report more positive on organizational readiness and more positive implementation outcomes.
Study design and data collection
This study is a correlational implementation study performed alongside a two-armed RCT (21). The combined study can be described as a hybrid effectiveness-implementation type 2 trial (43, 44), although the implementation strategies are not tested per se. To test the five research hypotheses, we will include a quantitative longitudinal web-based data collection spanning three time points, among SPARCK practitioners and their leaders. The first time point (T1) is during the initiation of the intervention in 2023, shortly after training of practitioners and information dissemination to leaders. The second data collection (T2) is after approximately one year when the practitioners have completed two cases each, and the last data collection (T3) after approximately two years, when the practitioners have completed four cases each. In addition, for hypotheses one, two, and four concerning relations with clinical outcomes, we will also include parent reported data on child symptoms and parent-child interaction from the RCT at pre, post (treatment termination), and follow-up (six months after post) (21).
Setting, participants, and recruitment
The study will be conducted in 24 municipalities, which vary in size, demography and urbanicity and together represent all five health regions in Norway. Efforts are made to recruit heterogeneous municipalities, to reflect conditions as they are in the real world. The municipalities are part of an implementation network hosted by Norwegian Center for Child Behavioral Development (NUBU; Norwegian acronym), and thus have some experience with implementation of EBIs (45). Of the 24 municipalities, seven participated in co-creation of SPARCK (19).
Participants in the implementation study are SPARCK practitioners and municipal leaders. Currently, this includes 44 recruited practitioners, including 14 from the seven co-creation municipalities. In addition to training in SPARCK, all practitioners are trained in and actively practice Parent Management Training – Oregon model (PMTO) (45, 46). Some are also trained in other EBIs. All practitioners are employed in municipal frontline mental health services, such as health care, school health services, and child welfare services. Participants also include the practitioners’ leaders, including their immediate manager, and superior leaders at mid and top managerial levels in the various services. Around 80 municipal leaders have so far been identified and recruited to the study. However, as staff turnovers are bound to happen, more will be recruited as needed. Written informed consent will be obtained from all participants before inclusion in the study.
Implementation frameworks and theory
The study will be guided by the determinant framework CFIR (30, 31), which will be used to systematically specify and assess potential barriers and facilitators along its domains (innovation, outer setting, inner setting, individuals, and implementation process) with underlying constructs and to explain how these determinants influence the implementation outcomes. The implementation theory Organizational Readiness (38) will provide a theoretical underpinning for how readiness is related to implementation outcomes. Proctor et al. (5) provide a framework to evaluate the four key implementation outcomes. Lastly, the Implementation Research Logic Model (IRLM) (47) will be used as a tool for reporting and specifying how the various elements (i.e., determinants, strategies, mechanisms of action, implementation outcomes, and intervention outcomes) are believed to relate.
Measures
All measures in the current study are Norwegian versions, which have been adapted to the current context with permissions from the original authors and have undergone piloting with similar individuals to the study sample. Measures on clinical outcomes are described in detail in the RCT protocol (21), and include various standardized and widely used instruments to investigate parent reported change in child externalizing and internalizing symptoms.
Background variables. Demographic and service-related variables will be obtained from all participants (i.e., age, gender, municipality, type of employment service, educational level, professional background, role, job experience, training in EBIs, counselling experience, familiarity with SPARCK etc.).
Fidelity to SPARCK. To investigate hypothesis one, Fidelity Questionnaire (FidQ) will be used to measure practitioners’ fidelity to SPARCK. FidQ was developed during the development phases of SPARCK, detailing recipients, intervention strategies employed, pedagogical tools, client engagement, and parental assessments of goals in intervention. FidQ will be completed by SPARCK practitioners weekly after each session.
Implementation outcomes. Hypotheses two, three and four will be tested using various measures of implementation determinants and outcomes. In addition to fidelity, the implementation outcomes that will be tested are acceptability, appropriateness, and feasibility (5, 35). These will be measured using the 4-item scales Acceptability of Intervention Measure (AIM), Intervention Appropriateness Measure (IAM), and Feasibility of Intervention Measure (FIM) (48). AIM, IAM, and FIM are measured on a 5-point Likert scale from 1 (completely disagree) to 5 (completely agree). Hence, higher scores indicate greater acceptability, appropriateness, and feasibility respectively.
Implementation determinants. The 10-item version of Organizational Readiness for Implementing Change (ORIC) will be used to determine a collective level of organizational readiness for change (37). The scale measures how well employees at an organization feel they can implement the change in processes required by a proposed intervention and the degree to which they are likely to initiate change, exert greater effort, exhibit greater persistence, and display more cooperative behavior. The measure consists of two subscales labelled Change Commitment (5 items) and Change Efficacy (5 items). Items are measured on a 5-point Likert scale from 1 (Disagree) to 5 (Agree). Sum score is calculated across all items. Lower scores represent less organizational readiness for implementing change; higher scores represent a more favorable organizational readiness for implementing change.
To measure other implementation determinants, we will use Implementation Determinants Measure (IDM). It consists of 47 items assessing factors that may hinder or foster successful implementation of an intervention into a given context. The measure is based on a 40-item scale, which was partly founded on CFIR (30) and partly on qualitative interviews with people involved in implementing mental health promotion programs in schools and kindergartens in Norway (32–34). For the present study, IDM was adapted and customized to fit SPARCK and the municipal services in Norway, as well as the current study’s target problem areas. The measure includes seven items not included in the 40-item version. The adaptations and additional items cover findings gained from the semi-structured interviews with 14 SPARCK practitioners and 15 municipal leaders who used SPARCK in the two test cycles (19, 27). Additionally, they cover items from pilot tests of the adapted version among relevant municipal practitioners and leaders. Importantly, we made an effort to reflect the updated CFIR 2.0 (31) and an abbreviated measure of the CFIR-based Barrier Buster Tool when modifying and adapting it. The latter, developed by the CFIR Research Team in Ann Arbor, Michigan (based on unpublished work), assesses the constructs most reported to be associated with implementation outcomes. The final 47-item measure covers the CFIR domains innovation (14 items), outer setting (4 items), inner setting (8 items), individuals (8 items), and implementation process (13 items). Together, these comprise several sub constructs (i.e., innovation relative advantage, innovation complexity, individuals’ attitudes, skills and motivation, external pressure, tension for change, implementation climate, available resources etc.). Items are measured on a 7-point Likert scale from 1 (strongly disagree) to 7 (strongly agree). After reversing negatively phrased questions, a sum score will be calculated across all items. Lower scores indicate less positive environment for successful implementation; higher scores indicate more positive environment for successful implementation. In addition, one open ended question is included (“Can you think of other factors that may affect the effectiveness or implementation of the intervention (positively or negatively)?”). After the study period, the IDM will also be used to detect barriers within and across organizations and municipalities, to guide selection and/or tailoring of relevant implementation strategies if SPARCK is implemented and scaled up. Subsequently, data from the IDM will be used in a future study to identify and understand relations between implementation strategies and mechanisms of change (49).
Implementation strategies. Implementation strategies used during the clinical testing of SPARCK will be specified and reported throughout the study period (see Additional file 1 for a list of initiated and planned implementation strategies), along with reporting of and justification for tailoring of strategies. Strategies will be named and defined to align with the Expert Recommendations for Implementing Change (ERIC) compilation (25) and Proctor et al.’s (39) recommendations for specifying and reporting implementation strategies. Choice and adaptation of implementation strategies are based on NUBU’s implementation infrastructure and experience with implementing other EBIs (45), relevant implementation theories, models, and frameworks (26, 30, 31, 38), as well as data from the contextual analysis of barriers and facilitators to implementation in the development and optimization phase of the SPARCK project (27). After study completion - and if SPARCK will be implemented into regular practice, there will be a follow-up study in which stakeholders will be asked to aid in choice and tailoring of appropriate implementation strategies to address barriers and facilitators, as well as investigation of implementation mechanisms (49).
Data analysis
The setting of the current study is inherently multilevel and hierarchical in structure. Participants work in different service settings across municipalities, and the municipalities themselves differ greatly in terms of geography, size, and resources. As such, we assume the need to address the multilevel context and will use multilevel linear modelling to assess the effects of time and implementation determinants on clinical effectiveness and implementation outcomes (50, 51). In addition, we will assess demographic factors, moderators, mediators, and associations relevant to the proposed hypotheses. The project builds on a co-creative optimization process, which substantially enhances the potential for positive effect and implementation. Thus, we will also investigate whether there are any effects between municipalities that were part of the co-creation process and those who were not.