The Sustainment and Adaptation of an English School Intervention to Improve Behaviour, Health and Wellbeing: a Qualitative Longitudinal Case Study

doi:10.21203/rs.3.rs-892545/v1

Download PDF

Research

The Sustainment and Adaptation of an English School Intervention to Improve Behaviour, Health and Wellbeing: a Qualitative Longitudinal Case Study

https://doi.org/10.21203/rs.3.rs-892545/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Background: The sustainment of school-based interventions to improve students’ behaviour, health and wellbeing has been relatively unexplored compared to healthcare interventions. Discontinuing effective interventions prevents new practices from reaching new student cohorts and wastes implementation investment. This study examines the sustainment and adaptation of a school intervention to improve risk behaviours that aimed to enhance sustainability by encouraging school commitment, ownership and local adaptation.

Method: A case study was conducted of the sustainment of ‘Learning Together’, an intervention to reduce multiple risk behaviours initiated in English secondary schools through an effectiveness trial. The intervention entailed restorative practice (RP), a staff-student action-group and a curriculum. Qualitative, longitudinal data were collected from five schools: interviews with multiple staff per school, and with students and external facilitators the first-year post-trial; interviews with one staff member per school two years post-trial; and descriptive data from the trial’s original process evaluation.

Results: Learning Together, as a whole intervention, was not sustained two years post-trial. RP was the most successfully sustained component; all staff interviewed continued to use RP in some form in their individual practice in years 4 and 5 and was sustained at school-level in one school. The curriculum and action-groups were discontinued in all schools, though actions initiated by the groups were sustained in two schools. Staff were motivated to sustain an approach at an individual-level if they perceived it as more effective than existing practices at improving students’ wellbeing or behaviour and it was practically achievable. Elements designed to build commitment, ownership and local adaptation were largely ineffective at sustaining components, as sustainment at school-level required ongoing attention to changes to organisational practices, policies and systems, which was beyond the remit of individual staff.

Conclusion: Schools need greater support from intervention developers to sustain interventions at school-level. Adaptation could help or hinder sustainment; schools need support to adapt components according to their theoretical rationale and develop different implementation options. Further methodologically strong primary research on sustainment and sustainability strategies is needed.

Health Economics & Outcomes Research

Health Policy

sustainment

sustainability

interventions

adaptation

Previous evaluations of the sustainment of school health interventions have used weak designs and have not examined interventions designed to be sustainable.
This study used qualitative longitudinal research to examine accounts of the sustainment of a whole-school intervention designed to maximise school commitment, ownership and local adaptation to enhance sustainability.
The intervention was partially sustained in the two years post-implementation. Adaptation could help or hinder sustainment and was influenced by components’ perceived effectiveness/value compared to existing practices.
Considering a theory of sustainment from implementation outset and developing and evaluating sustainability strategies would advance knowledge of sustainment processes.

In the last thirty years, there has been a surge in effectiveness trials of school-based health interventions (1–6). Research has shown that school interventions can effectively improve childhood health and wellbeing, with associated benefits for educational achievement (3, 7–10).

However, new interventions must be able to function within existing delivery systems and require people to work individually and collectively to sustain them (11, 12). Theory and evidence from health and other sectors suggests that a lack of support to maintain an innovation can be a disruptive experience for those delivering it, who have made considerable efforts to learn new skills or approaches, and could negatively impact on their willingness to pilot new interventions in the future (13–17). To tackle pervasive health problems like childhood obesity or poor mental health through school interventions, effective interventions need to be sustainable for schools so that further cohorts of students can benefit from them (18). However, few empirical studies have examined the sustainment of school health interventions (12, 19).

The current status of sustainability research

Overall, existing evidence relating to schools is limited; a review of sustainment identified only 24 studies of 18 different interventions (19). Most evaluations used cross-sectional quantitative or mixed-methods designs; none drew on qualitative longitudinal data to explore teachers’ or students’ accounts of how sustainment emerged over time. None examined interventions designed for sustainability. Evidence to date from school settings mirrors that of healthcare indicating that partial sustainment, the sustainment of some components by some schools or staff, is the most common outcome (19, 20).

Differences in how studies have defined, conceptualised and assessed sustainment, and gaps in reporting make it difficult to accumulate learning (19–21). The three most common defining sustainment characteristics studied are: 1) the continuation of intervention activities; 2) the maintenance of health benefits, and/or 3) building a community/organisation’s capacity to develop and deliver the intervention (20, 22, 23). Over the last decade, there has been a gradual shift from considering deviations from the original intervention protocol to be an implementation failure (24, 25), towards thinking of adaptations as a characteristic of or precursor to sustainability (23, 26). Adaptations which alter the form of an intervention but are faithful to its theory of change could, for example, improve an intervention’s effectiveness for cohorts of students with different needs or improve staff ownership of an intervention (16, 27, 28). However, the evidence on how adaptation affects sustainability is weak with most studies failing to describe adaptations or examine their impact on outcomes (20, 29).

Multiple facilitators and barriers have been found to influence sustainability in school settings. Key facilitators include: support from school executives/administration; staff self-efficacy and belief in the intervention’s value; staff observing a positive impact on students’ engagement and behaviour; and regional/national health policies that legitimised health promotion (19, 29). Barriers include: insufficient funding and material resources, staff turnover, a lack of training for new teachers or booster training (19, 29). The degree to which an intervention can be shaped into existing school practices and routines can influence the fidelity with which practices are sustained (19, 30). However, many studies have been assessed as methodologically weak, with a lack of longitudinal studies following sustainability from initial implementation and few studies from outside of the USA (19, 29).

This study will contribute to gaps in the literature by drawing on qualitative longitudinal research to examine the sustainment of ‘Learning Together’, which was introduced into English secondary schools through a three-year effectiveness trial, and was found to be effective in reducing bullying and improving students’ health and wellbeing (31). Informed by existing literature on sustainability (11), the intervention was unusual in being designed to enhance sustainability by building schools’ commitment to change, encouraging school ownership of the intervention, and allowing local adaptation using standardized processes (see method). Sustainment is defined in this study as the continuation or discontinuation of the intervention two years after external funding and/or other resources to initially implement the intervention end (in this case, the end of the trial), and sustainability is defined as the likelihood of sustainment. The study asks: how did staff perceive the effectiveness of each core component?; how did staff and students describe the sustainment and adaptation of Learning Together in the two years following the trial?; and how did adaptation affect the intervention’s sustainability? A detailed account of barriers and facilitators to its sustainment will be reported separately.

A single-case, multi-site study design was used to explore ongoing implementation in context over time. The study was approved by the London School of Hygiene & Tropical Medicine Research Ethics Committee (reference number 14223).

Description of the case of ‘Learning Together’

Learning Together was a whole-school intervention to reduce multiple risk behaviours. It was evaluated through a three-year cluster randomised controlled trial, which has been described in detail elsewhere (31, 32). The trial ran from 2014 to 2017 in 40 state secondary schools in south-east England, 20 per arm. Schools were not aware of the trial results in the first-year post-trial when fieldwork for this study took place. The results were published in the autumn of the second-year post-trial and found that the intervention had positive impacts on bullying, mental health and wellbeing, psychological functioning, quality of life, smoking, alcohol and drug use, drunkenness and police contacts compared to controls at 36 months (31). The intervention had three main components: restorative practice (RP), action-groups and locally-decided actions, and a SEL curriculum (see Table 1 for a description of its components and Fig. 1 for the trial’s theory of change).

Table 1

The core components of Learning Together
Restorative practices (RP)	• In the trial’s first year, all school staff received 2–3 hr of training in RP approaches, using respectful language to challenge behaviour and strengthening relationships (33). • A further 3-day training was provided for 5–10 staff selected by schools to deliver restorative conferences, more formalised meetings to address instances of bullying, aggression or wrong-doing (33).
Action-groups and locally-decided actions	• Schools formed an ‘action-group’ comprising at least six students and six staff which met six times per year. The group’s objectives were to: o review data on student health needs and views about the school from an annual student survey (carried out by the trial team); o decide local actions to address the needs identified, including how RP was to be used within the school; o review and revise relevant school policies to ensure that these supported an inclusive and restorative school environment; and o oversee the implementation of the social and emotional learning (SEL) curriculum. • Schools were asked to recruit diverse students, including those prone to disengagement, and groups had to include a member of the senior leadership team (SLT). • For the trial’s first two years, action-groups were supported by a trained external facilitator.
SEL curriculum	• Schools were provided with lesson plans and slides to guide teachers’ delivery of 5–10 hours per year of a SEL curriculum for students from year groups 8–10 (age 12–15). • The curriculum was designed to complement schools’ existing personal, social, health, and economic (PSHE) education provision. Schools were expected to deliver a minimum of 5 hours per year.

Learning Together was purposively selected as the case as it was a complex intervention, comprising whole-school and curriculum components, and it was designed to encourage sustainability. It aimed to build schools’ commitment to change by drawing on local data of students’ needs and, by creating a staff-student group to lead, co-ordinate and adapt the intervention according to local needs, it aimed to augment schools’ existing good practice, heighten the intervention’s perceived relevance and acceptability, and develop school ownership. Its design was influenced by Hawe et al.’s (28) proposition that the function of intervention components could be standardised while the adaptation of their form could improve local responsiveness. Fidelity of form and function was intended for the: annual need assessment, RP training, SEL curriculum, and the process of decision-making via the action-groups. Fidelity of function but not necessarily form was intended for: ensuring that revisions to school policies and rules tied in with existing work; choosing which curriculum units to deliver; implementing RP approaches; and other locally decided actions aiming to improve relationships and student participation (33).

Five schools were selected for diversity in implementation fidelity in the trial’s final year; the assumption was that this would lead to variation in sustainment over the next two years (see (33) for details on how fidelity was measured). Despite a good level of fidelity in years 1 and 2 (see additional file 1), in the trial’s third year fidelity dropped, particularly for the curriculum (31). Schools in the intervention arm were organised into five categories based on the quality of year 3 implementation, ranked from high implementation through to complete discontinuation; one school was selected per category (see Table 2, schools and participants have been given pseudonyms). Data were gathered on schools’ characteristics and schools were sampled to ensure as much diversity as possible within the trial’s schools. All trial schools had achieved a ‘good’ or ‘outstanding’ rating from Ofsted (schools ‘requiring improvement’ were excluded from the trial).

Table 2

Case study schools’ characteristics
Category: School	Level of implementation during year 3	Urban/ suburban	Single/mixed	Size	% FSM (past 6 years)
A: Downton Park	Implemented RP, six action-groups, and the curriculum.	Suburban	Single	> 1250	10–25%
B: Franklyn	Implemented RP, an adapted form of action-groups conducted weekly, and the curriculum.	Urban	Mixed	750–1000	> 50%
C: Fern Grove	Implemented RP and action-groups at a lower dose, and discontinued the curriculum.	Urban	Mixed	1000–1250	> 50%
D: Bletchford	Implemented RP and discontinued the action-groups and the curriculum.	Suburban	Mixed	> 1250	25–50%
E: Greenthorne	Discontinued RP, action-groups and the curriculum.	Suburban	Mixed	> 1250	< 10%

Data collection

Data collection for the case study was both retrospective and prospective, asking participants about their previous involvement in Learning Together during the trial and their current views and experiences of working with the intervention. Table 3 summarises the timeline of the trial and this study.

Table 3

Timeline for case study
	Intervention year	Trial/post-trial
2014/15	Year 1	Trial
2015/16	Year 2	Trial
2016/17	Year 3	Trial
2017/18	Year 4	Post-trial case study
2018/19	Year 5	Post-trial case study

Trial data (Years 1 to 3)

Process evaluation data from the trial were used to describe how the intervention had been implemented in years 1 to 3: interviews with staff action-group members in year 2, an interview with a SLT member in year 3 and facilitator interviews in year 2. These data were used to inform the interview prompts and describe implementation during the trial. The accuracy of the details was checked with participants during interviews. The data were not formally analysed, with the exception of data relating to the curriculum which was sought out specifically. The curriculum was discontinued in three schools during the trial; subsequently, more comprehensive recall of reasons for discontinuation could be found in year 2 and 3 interviews, rather than relying on retrospective accounts.

One-year post-trial (Year 4)

Twenty-five semi-structured interviews were conducted between January and March 2018: 18 with school staff, 4 with students and 3 with external facilitators. Table 4 presents participants’ characteristics. Staff interviews were conducted in person (n = 14) or by phone (n = 4); interviews with students and facilitators were conducted in person.

School staff who had been/continued delivering the intervention were identified by snowball sampling, beginning with staff who had led Learning Together in each school. One teacher who was the intervention lead in Bletchford in years 1–2 declined to participate and one teacher who was the intervention lead at Fern Grove in years 1 and 3 was on leave and could not be interviewed. Interviews with staff aimed to clarify implementation in year 3 and elicit: whether/how components were delivered in year 4 and their involvement in its delivery; barriers/facilitators to sustaining the components; whether/how the components were integrated within school systems; whether/how the intervention affected their attitudes, skills/knowledge or relationships with other staff members or students.

For the three schools that had continued the action-groups in year 3, the lead teacher was asked to identify two students who may wish to participate in the study. Four student interviews were conducted from two schools (see Table 4). One school (Downton Park) declined to invite any students due to exam pressures. Interviews with students aimed to: clarify how the action-group was implemented in year 3; whether/how the action-groups continued in year 4; and whether it was integrated with other school systems. School behaviour and anti-bullying policies for year 4 (2017/18) were collected from school websites.

Two-years post-trial (Year 5)

The most senior staff-member interviewed in year 4 was invited to interview. Five semi-structured interviews were conducted in June 2019 (three in person, two by phone), one per school, with the exception of Fern Grove school where the identified staff-member had left the school and a group interview was conducted with a previous interviewee and another senior leader who led the action-groups and was not interviewed in year 4. All interviews were audio-recorded and transcribed. School behaviour and anti-bullying policies for year 5 (2018/19) were collected from school websites.

Table 4

participants’ characteristics (pseudonyms are used for schools and participants)
School	Participant	Role 1st yr post-trial	Action-group member (NB groups did not run in Bletchford and Greenthorne in Y3)	Participant attended in-depth training in RP	Gender	No. of years at the school at 1st yr post-trial	Year group (if applicable)	Interview 1st yr post-trial	Interview 2nd yr post-trial
Downton Park	Angela	Teacher	Yes	Yes	F	> 12	-	√
Downton Park	Callum	Teacher	Yes	Yes	M	< 5	-	√	√
	Victoria	Teacher	Yes	Yes	F	5–8	-	√
	Rachel*	Facilitator	Yes	n/a	F	-	-	√
Franklyn	Matt	Senior leader	Yes	No	M	5–8	-	√	√
	Gregory	Teacher	Yes	Yes	M	5–8	-	√
	Jessica	Pastoral	Yes	Yes	F	5–8	-	√
	Amelia	Teacher	Yes – Y2 only	Yes	F	> 12	-	√
	Craig	Student	Yes	n/a	M	-	Year 11	√
	Sara	Student	Yes	n/a	F	-	Year 11	√
	Miriam⁺	Facilitator	Yes – Y1 and 2 only	n/a	F	-	-	√
Fern Grove	David	Senior leader	Yes	Yes	M	5–8	-	√
	Harriet	Senior leader	Yes – Y1 and 3 only	No	F	5–8			√
	June	Pastoral	Yes	No	F	9–12	-	√	√
	Katie	Teacher	Yes	Yes	F	< 5	-	√
	Harry	Student	Yes – Y2 and 3 only	n/a	M	-	Year 10	√
	Kristen	Student	Yes – Y2 only	n/a	F	-	Year 9	√
Bletchford	Joe	Senior leader	No	No	M	> 12	-	√	√
	Brett	Teacher	Yes – Y2 only	No	M	9–12	-	√
	Jenny	Teacher	Yes	Yes	F	9–12	-	√
	Penny	Pastoral	Yes – Y2 only	Yes	F	> 12	-	√
	Richard	Facilitator	Yes – Y1 and 2 only	n/a	M	-	-	√
Greenthorne	Colin	Senior leader	Yes – Y2 only	No	M	> 12	-	√	√
	Amy	Teacher	Yes	Yes	F	9–12	-	√
	Toby	Teacher	Yes	Yes	M	> 12	-	√
	Paul	Teacher	Yes – Y1 only	Yes	M	> 12	-	√
Also facilitator for Greenthorne⁺Also facilitator for Fern Grove*

Data analysis

An a priori framework was used to code the sustainment and adaptation of the intervention, using each component as an organising theme. If a new set of resources and activities started in schools that resembled the intervention components (for example, new staff-student groups), an inductive appraisal was made of whether the new initiative was aligned with Learning Together’s theory of change (see Fig. 1) and whether staff believed the new initiative was generated because of the intervention or whether it would have happened regardless.

To understand staff perceptions of the components’ effectiveness, a thematic analysis was carried out on the primary data (34, 35). Interview transcripts were read and re-read to become familiar with participants’ accounts. Inductive, line-by-line coding was conducted using NVivo 12 software, labelling segments of the data. Each code’s data were checked for consistency of interpretation and re-coded as necessary. Constant comparison was used to explore how each theme manifested in each school, checking whether the explanation of each theme matched the data, actively checking for examples which did not fit (36). Deductive coding was carried out on the trial’s process evaluation data on the curriculum from year 2 and 3, coding the delivery of the curriculum and its perceived effectiveness, and its relationship to existing PSHE provision.

The study follows the standards for reporting qualitative research guidelines (see additional file 2). This study was conducted a part of PhD scholarship (see (37) for a reflexivity statement).

None of the schools sustained Learning Together in its entirety; parts of the intervention were sustained in some schools (see Table 5). The next sections examine staff perceptions of each component’s effectiveness and its sustainment (or not) and adaptation.

Table 5

Summary of staff-reported sustainability events from years 3 to 5
School	Year 3 (last year of trial)	Year 4	Year 5*
Downton Park In years 1 and 2 of the trial, implemented RP, six action-groups, and the curriculum.	RP • RP used in staff student mentor training and student peer mentor training, newly qualified and support teachers received RP training. • RP used in tutorial time. • RP principles taught in one PSHE lesson for Year 9 students. • RP principles discussed in assemblies. • Staff trained in-depth continued to use RP. Action-groups and actions • Action-groups continued to be held. Curriculum • Curriculum delivered in tutorial time (integrated with RP).	RP • Newly qualified and support teachers continued to be trained in RP. • Staff uncertain whether RP used in tutorial time. • RP principles taught in one PSHE lesson for Year 9 students. • RP principles discussed in assemblies. • Staff trained in-depth continued to use RP. Action-groups and actions • Action-groups discontinued. Curriculum • Staff uncertain whether curriculum continued to be used in tutorial time.	RP • Staff trained in-depth continued to use RP. Action-groups and actions • A new action-group was created that aligned with LT’s Theory of Change (ToC) but was likely to have occurred without the intervention. Curriculum • Curriculum discontinued.
Franklyn In years 1 and 2 of the trial, implemented RP, an adapted form of action-groups, and the curriculum.	RP • RP used in discipline process in ‘RED’ meetings. • Staff trained in-depth continued to use RP. Action-groups and actions • Action-groups discontinued. • Two new action-groups created that aligned with LT’s ToC and may have occurred in part because of the intervention. Curriculum • Curriculum delivered in tutorial time.	RP • RP used in discipline process in ‘RED’ meetings. • Staff trained in-depth continued to use RP. Action-groups and actions • New action-groups continued, reviewed school policies, organised events and influenced student and staff training. Curriculum • Curriculum discontinued.	RP • RP continued to be used in discipline process in ‘RED’ meetings. • Staff trained in-depth continued to use RP. • Staff received external training in student resilience that had embodied the principles of RP. Action-groups and actions • New action-groups continued, organised events and influenced student and staff training. Curriculum • Curriculum discontinued
*NB, the data in this column is based on one staff members’ report

Fern Grove

By year 2 of the trial, implemented RP and action-groups at a lower dose, and discontinued the curriculum.

· Staff trained in-depth continued to use RP.

Action-groups and actions

· Action-group continued.

· Fern Grove’s “rules of conduct (ROC)” continued to be used.

Curriculum

· Curriculum discontinued.

· Staff trained in-depth continued to use RP.

Action-groups and actions

· Action-group discontinued.

· Three new action-groups created, two of which aligned with LT’s ToC and were considered an evolution of the intervention.

· A new student voice coordinator was appointed.

· Fern Grove’s ROC continued to be used.

Curriculum

· Curriculum discontinued.

· Staff trained in-depth continued to use RP.

Action-groups and actions

· New action-groups discontinued.

· Student voice coordinator remained in post.

· Fern Grove’s ROC continued to be used.

Curriculum

· Curriculum discontinued.

Bletchford

By year 2 of the trial, implemented RP and had discontinued the action-groups and the curriculum.

· RP training rolled-out across the school.

· Staff trained in-depth continued to use RP.

Action-groups and actions

· Action-group discontinued.

Curriculum

· Curriculum discontinued.

· RP integrated into the school as part of a new discipline system called Education for All. Staff trained in using the new system.

· Staff trained by external or internal trainers used RP.

Action-groups and actions

· Action-group discontinued.

Curriculum

· Curriculum discontinued.

· RP integrated into the school as part of Education for All. Staff trained in using the new system.

· Staff trained by external or internal trainers used RP.

Action-groups and actions

· Action-group discontinued.

Curriculum

· Curriculum discontinued.

Greenthorne

By year 2 of the trial, discontinued RP, action-groups and the curriculum.

· Staff trained in-depth continued to use RP.

Action-groups and actions

· Action-groups discontinued.

Curriculum

· Curriculum discontinued

· Staff trained in-depth continued to use RP.

· RP was written into discipline procedures.

Action-groups and actions

· Action-group discontinued.

Curriculum

· Curriculum discontinued.

· Staff trained in-depth continued to use RP.

· RP remained written into discipline procedures.

Action-groups and actions

· New action-group created that aligned with LT’s ToC and may have occurred because of the intervention.

Curriculum

· Curriculum discontinued.

The perceived effectiveness of RP; its sustainment and adaptation

RP was the most highly regarded component of Learning Together across the schools; most staff interviewed had attended in-depth RP training. Staff had been motivated to sustain RP approaches and had appraised it as an effective way of drawing students into conversations about behaviour, de-escalating conflict, and improving students’ behaviour and their own relationships with students:

“RP was the starting point to getting staff, including everybody, even myself included, to think about how our... our actions and reactions are actually the central cause of whether it's going to be a positive outcome or a worse outcome [for the incident].” Joe, SLT, Bletchford, year 4

Unlike the punishment of detentions, which were rarely administered by the staff members who had set them, RP brought together the parties involved in the incident and gave students the opportunity to reflect on another student’s or staff-member’s perspective, and develop understanding and skills in how to maintain better relationships.

RP was also the most successfully sustained component of Learning Together; all staff interviewed continued to use RP in some form in their individual practice in years 4 and 5. RP staff training took place in all schools and local RP approaches were implemented during the trial. However, the degree to which the approach was embedded across each school varied greatly.

Bletchford was the most successful school at sustaining RP. In trial years 1–3, training was cascaded across the whole school community, including senior leaders and teaching staff (through continual professional development (CPD)), auxiliary staff and students. In year 3, a group of staff were trained to be RP facilitators to help deliver the CPD programme.

“We trained the whole staff. I mean the whole staff... We invited the dinner ladies, we invited the office staff, we... and that really is the sort of philosophy behind it; if you're going to be a restorative school, everybody has to sign up to it.” Jenny, staff, Bletchford, year 4

Towards the end of the trial, Joe, a senior leader, employed a consultancy, Education for All (E4A)¹, to provide a framework and set of steps to help embed RP into Bletchford’s discipline procedures. All Bletchford staff reported that the principles of E4A aligned with RP. In years 4 and 5, the school continued to rolled-out E4A’s discipline approach: a staff working group for behaviour was created to oversee the integration process of E4A, the CPD programme continued to focus on RP/E4A principles and E4A trainers delivered coaching to members of staff who needed extra support.

Downton Park predominantly focused on teaching students RP principles during the trial. In years 1–3, form tutors were trained to teach the principles of RP to students during tutorial/registration time, a PSHE day on RP and student peer mentoring training was held. In years 3 and 4, RP was discussed in student assemblies but in year 4, its use in tutorial time petered out. RP was diffused to some staff-members but not the whole school: in years 3 and 4, newly qualified teachers and support staff received RP training. RP was not written into discipline procedures though it did appear in the school’s anti-bullying policy. In year 5, the teacher interviewed thought that that the principles of RP still permeated the school but restorative conferences were not held and RP training was needed for new and existing staff.

In Franklyn, during trial years 1–3, RP was not cascaded to staff beyond the training provided by the trial. In year 3, RP was written into the school’s behaviour policy. Students who took part in a serious behavioural incident had to participate in a ‘RED²’ meeting, a restorative conference between the teacher and student. In years 4–5, RED meetings continued but staff-members trained in RP thought that the meetings were not always held according to RP principles, having witnessed staff using the meeting as punishment. In year 5, Matt, a senior leader reported that consistency of practice remained an issue.

At Fern Grove, many of the staff who had received in-depth RP training had left the school by the end of the trial’s second year. In year 3, staff reported that they applied restorative conferencing when holding post-exclusion meetings with students and parents (to reintroduce them back into school). RP principles were mentioned in Fern Grove’s behaviour policy in year 4 but it was not written into procedures for dealing with behaviour. In year 5, the roles of senior staff were revised separating staff responsibilities for educational inclusion and student behaviour, in contrast to a RP approach which unites inclusion and behaviour. There was a plan for further training on RP but staff-members interviewed said that RP was not embedded in behaviour systems:

“I think the commitment to restorative practice is there. I just don’t think it’s the practice itself is embedded.” Harriet, SLT, Fern Grove, year 5

In Greenthorne, one additional RP training session was held for staff-members in year 1 and one staff-member had created a summary of the RP approach for staff that was not utilised. There were no further developments in the diffusion of RP to staff or students in Greenthorne in years 2 and 3. In year 4, RP was written into discipline procedures and in year 5, the SLT staff-member interviewed was keen to embed RP further:

I guess it’s on the agenda and we’re fully aware of it, but it’s never kind of been formalised, or it’s never been taken on as a whole-school approach, or anything like that... Colin, SLT, Greenthorne, year 5

The perceived effectiveness of the actions groups and local actions: discontinuation, sustainment and adaptation

There was a divide between schools regarding staff perceptions of the action-groups’ effectiveness. Staff at Greenthorne, Downton Park and Bletchford did not use the action-groups to revise school rules and policies and considered their existing student councils to be effective for raising students’ concerns with SLT. At Greenthorne, the external facilitator and staff reported that the groups had failed to engage students in decision-making to create purposeful, meaningful actions. In Downton Park and Bletchford, students’ attendance was intermittent and students were only superficially involved in decision-making. The action-groups had mainly been used to discuss the implementation of RP and did not focus on other ways of promoting students’ learning and connection to the school community:

“The meetings were purely about...embedding restorative practice and... finding ways of... communicating better and so forth. But... I don't know if they needed to run... given the impact.” Angela, staff, Downton Park, year 4

In contrast, staff from Fern Grove and Franklyn described their experiences of the action-groups as powerful, transformative or challenging. Staff at the two schools valued the action-groups because they were different from existing school councils, involving a diverse range of students with different views and experiences of being disciplined, and had created a forum for building trust between staff and students that were disengaged from school. This diversity of student experiences had been prized when revising school rules and policies on behaviour:

“One thing I do think is that the school will never go back to... implementing things like behaviour policies without student consultation.” David, Fern Grove, year 4

By the end of year 3, the action-groups were discontinued in all five schools. Greenthorne and Bletchford schools discontinued the groups at the end of year 2. Local actions decided by the groups at Greenthorne and Bletchford to improve student’s behaviour and connection to schools were not sustained beyond year 2: a peer-mentoring programme at Bletchford and the lunchtime drop-in for students at Greenthorne were dropped. Downton Park discontinued the groups at the end of the trial.

Fern Grove continued to run the action-groups in year 3 but with a new cohort of students “just to give a different...a new voice as well” (June, Year 4). David and June commented that the group did not have a clear direction in year 3 and the group was discontinued at the year’s end. In year 5, the two staff-members interviewed said that the groups had needed a project on which to focus, otherwise their remit was too similar to the school council. At Franklyn and Fern Grove, several local actions developed in trial years 1–2 were sustained into year 5. At Franklyn, the group introduced: a more severe sanction for late attendance; an award assembly for students; the RED meeting; and the Franklyn pyramid, a poster in every classroom clarifying the school’s stepped response to behaviour. At Fern Grove, the group contributed to a change in the reward system for students and, with staff, co-developed a classroom ‘Rules of Conduct’. At both Franklyn and Fern Grove, staff and students reported that the actions had helped to clarify and improve behaviour policies. However, achieving buy-in and consistency from all staff-members remained an ongoing issue.

Four schools created new staff-student groups that aligned with Learning Together’s theory of change. At the end of year 3, Franklyn created two new groups on Lesbian Gay Bisexual and Trans (LGBT) and equality issues led by a staff-member who had been an action-group member and several student action-group members participated in the new groups. The groups were sustained in years 4 and 5. David, a senior leader at Fern Grove, created three new groups in year 4 involving a diversity of students to consult with on policies about teaching and learning, behaviour and wellbeing, and student voice. Students from the action-groups were invited to participate in the student-voice group. The new student-staff groups ran for year 4 only and were not sustained once David left the school. Senior leaders from both Franklyn and Fern Grove reported that they saw the new groups as an evolution of the action-groups, though this view was not shared by other staff and students. In year 4, both schools also created a new funded position to focus on student voice and engagement, positions which were sustained into year 5.

Downton Park also created a new group focused on LGBT and equality in year 5, though no connection was made by staff between this group and the previous action-groups. Greenthorne also started a new student-staff group in year 5. Colin, a senior leader, started an ‘attitude-to-learning’ group, comprised of 12–15 students, many deliberately selected for being disengaged from school, who were encouraged to shape behaviour policy. When asked whether the group would have been created regardless of Learning Together, Colin replied:

“Maybe in the back of my mind, considering we’d had the meetings with students and seeing the benefit of getting the student voice and the student ideas…. So maybe, maybe, that’s not a yes or no, but a maybe.” Colin, SLT, Greenthorne, year 5

The perceived effectiveness of the SEL curriculum: its adaptation and discontinuation

The curriculum was designed to be delivered through PSHE or another subject area of the school’s choice. However, existing PSHE provision differed greatly between schools and consequently, staff found it very difficult to timetable the curriculum or adapt it to the time available (see Table 6). Staff at Bletchford, Greenthorne, Franklyn and Fern Grove considered the curriculum inferior to schools’ existing PSHE provision.

Three schools discontinued the curriculum by the end of the trial (Bletchford, Greenthorne and Fern Grove); and Franklyn discontinued it in year 4. Staff at Downton Park gave positive feedback on the curriculum for disseminating RP approaches to students. However, they could not confirm whether it had been used beyond the trial when its delivery was transferred to staff that were not involved in Learning Together.

Table 6

Staff reported delivery of PSHE and the Learning Together curriculum in the five case study schools
	Existing PSHE provision for year group 8			Implementation of Learning Together curriculum			Sustainment of curriculum
	Regular lessons	Full days	Tutorial time	Regular lessons	Full days	Tutorial time	Sustainment of curriculum
Downton Park		One day per term.				Fully implemented in tutorial time (35 min) year 1–3 and in assemblies.	Uncertainty over implementation in year 4. Discontinued in year 5.
Franklyn	One lesson per week or fortnight*.			Partially implemented in lessons in years 1 and 2.		Fully implemented in tutorial time⁺ (unknown length) and in assemblies in year 3.	Discontinued in year 4.
Fern Grove			One 20 min tutorial a week, with occasional half-days on particular subjects.	Fully implemented in drama lessons in year 1.		Tutorial time (20 mins) in year 2.	Discontinued in year 3.
Bletchford	One lesson per week.			Partially implemented in PSHE lessons in year 1.		Partially implemented in tutorial time (20 mins) in year 2.	Discontinued in year 3.
Greenthorne	One lesson per fortnight.			Fully implemented in PSHE lessons (one in every four) in year 2.	Fully implemented in one full day in year 1.		Discontinued in year 3.
*Not clear from trial process evaluation⁺Tutor groups made up of mixed year groups.

¹Not real name

²Not real name

Summary of key findings

Although designed to maximise sustainability via elements aiming to build commitment, ownership and local adaptation, Learning Together in its entirety was not sustained: no schools sustained all three core components. Despite individual staff member’s commitment to using RP and the flexibility offered to schools in how they implemented RP approaches, schools struggled to deliver RP across the whole school. Only one school sustained the approach at school-level with the assistance of an external consultancy. Staff interviews indicated that the action-groups were not sustainable in their original format in any of the five schools beyond 2–3 years. However, action-groups in two schools created school-wide actions to revise school rules and behaviour policies which were sustained into years 4 and 5. The curriculum was discontinued in three schools by the end of the trial, in one school during year 4, and in the final school by year 5. The fate of Learning Together mirrored existing evidence on sustainment: some components were sustained by some staff in some schools (19, 20).

Although the action-groups were not sustained according to the definition used in this study, they may have met a capacity-building definition of sustainment (22). In three schools, staff created new student-staff groups in years 4 and 5 which aligned with Learning Together’s theory of change. Two of these schools created new funded positions to focus on student voice and inclusion. The groups changed the social bonds between staff members and students, and intervention principles and values were sustained (38–40).

Our findings suggest that a whole-school intervention is more difficult to sustain than a curriculum. The multiple components of the intervention were not sustained synergistically. Instead, staff chose to sustain only the intervention components that worked most successfully for them. Perceived effectiveness was the first necessity in gaining staff buy-in for sustainment. However, buy-in did not translate into knowledge or enactment of sustainment strategies at an organisational level. Staff at Fern Grove and Franklyn reported that, although the original action-groups had been valuable to initiate a process of change, other actions were needed to sustain changes that were beyond the remit of the group. These included providing new and refresher training in RP approaches, monitoring staff’s use of RP or new discipline policies, and achieving buy-in from all staff-members. The data suggest that schools need support to move the intervention from core group of interested individuals to a sustained school-wide, mainstreamed approach, which this intervention did not provide.

Learning Together aimed to encourage ownership of the intervention and responsiveness to local needs by allowing schools to adapt the form of some components. Learning Together’s design assumed that schools, with the support of the facilitator, would be able to work out contextually-relevant actions, building on Hawe et al’s (28) notion of maintaining ‘fidelity of function’. However, more precise guidance on adaptation was needed, alongside a greater understanding of how the components would relate to schools’ existing provision. The curriculum competed with existing PSHE provision and school staff’s time and ability to adapt the curriculum was low. Three schools thought their existing school council was sufficient for engaging students’ views and deciding how to implement RP approaches; they did not want to use the groups to revise school rules and policies and so created superficially relevant projects for the sake of fulfilling the trial’s requirements. In contrast, the two schools that valued the diversity of students in the action-groups (particularly those less engaged in school) decided on useful and relevant local actions that were sustained into year 5. The findings support the assertion that adaptations can be made as a response to contextual problems without regard to its theoretical rationale (41, 42), and can hinder as well as help sustainability, in contrast to established thinking that they are a necessary part of or precursor to sustainability (23, 26). The data indicate that schools needed to co-design viable options for adaptation that fit with their existing provision and timetabling needs, as well as the intervention’s theory of change.

Study limitations

No observations were conducted of RP being used in the schools or of the new student-staff actions groups to verify participants’ accounts; the findings are based on self-report data only with the exception of changes to behaviour policies which were examined directly. The schools participating all had good or outstanding Ofsted ratings so our findings may be less applicable to the sustainment of interventions in schools that have lower achievement and/or capacity.

Implications for research and policy

If we are to sustain effective interventions, developers/implementation agents need to consider sustainment from the outset and develop and evaluate sustainability strategies (43). Potentially, sustainability could be considered within theories of change or in separate theories of sustainment. Staff closely involved in implementation cannot be assumed to have the skills, knowledge or power to engage staff at an organisational-level to sustain an intervention, or know how to revise organisational policies or systems to integrate an intervention. Further sustainment support after implementation may be needed. More primary research that evaluates the sustainment of interventions beyond effectiveness trials and strategies to improve sustainment would advance the field.

Intervention developers should consider how they can best support sustainment and adaptation by asking participants about the perceived effectiveness and value of components relative to other (related) practices during early piloting, co-designing options for adaptation, simplifying intervention manuals and key messages about theoretical rationale for different components.

Learning Together, as a whole, was not sustainable, though RP had the potential to be sustained. This study highlights that practitioners delivering interventions need greater support from the intervention developers to harness organisational-level engagement, to sustain organisational-level knowledge of new approaches, and change organisational systems and policies. Further methodologically strong primary research on sustainment and sustainability strategies is needed.

CPD Continual professional development

LGBT Lesbian Gay Bisexual and Trans

PSHE Personal, social, health, and economic

RP Restorative practice

SEL Social and emotional learning

SLT Senior leadership team

Ethics approval and consent to participate

The study was approved by the London School of Hygiene & Tropical Medicine Research Ethics Committee, reference number 14223.

Consent for publication

Not applicable

Availability of data and materials

The datasets used and/or analysed during the current study are not publicly available to protect the anonymity of the schools and participants involved but anonymised data are available from the corresponding author on reasonable request.

Competing interests

The authors declare that they have not competing interests.

Funding

This study was funded through an Economic and Social Research Council PhD scholarship awarded to LH (ref. 1486173).

Authors’ contributions

LH analysed and interpreted the data and wrote the manuscript. CB contributed to planning the study, advised throughout the study process, and contributed to and commented on the manuscript.

Acknowledgements

Not applicable

Brown T, Summerbell C. Systematic review of school-based interventions that focus on changing dietary intake and physical activity levels to prevent childhood obesity: an update to the obesity guidance produced by the National Institute for Health and Clinical Excellence. Obes Rev. 2009 Jan;10(1):110–41.
Kriemler S, Meyer U, Martin E, van Sluijs EMF, Andersen LB, Martin BW. Effect of school-based interventions on physical activity and fitness in children and adolescents: a review of reviews and systematic update. Br J Sports Med. 2011 Sep 1;45(11):923–30.
Langford R, Bonell CP, Jones HE, Pouliou T, Murphy SM, Waters E, et al. The WHO Health Promoting School framework for improving the health and well-being of students and their academic achievement. Cochrane Developmental, Psychosocial and Learning Problems Group, editor. Cochrane Database Syst Rev [Internet]. 2014 Apr 16 [cited 2018 Jul 24]; Available from: http://doi.wiley.com/10.1002/14651858.CD008958.pub2
Denford S, Abraham C, Campbell R, Busse H. A comprehensive review of reviews of school-based interventions to improve sexual-health. Health Psychol Rev. 2017 Jan 2;11(1):33–52.
Axford N, Bjornstad G, Clarkson S, Ukoumunne OC, Wrigley Z, Matthews J, et al. The Effectiveness of the KiVa Bullying Prevention Program in Wales, UK: Results from a Pragmatic Cluster Randomized Controlled Trial. Prev Sci. 2020 Jul;21(5):615–26.
Chillón P, Gálvez-Fernández P, Huertas-Delgado FJ, Herrador-Colmenero M, Barranco-Ruiz Y, Villa-González E, et al. A School-Based Randomized Controlled Trial to Promote Cycling to School in Adolescents: The PACO Study. Int J Environ Res Public Health. 2021 Jan;18(4):2066.
Durlak JA, Weissberg RP, Dymnicki AB, Taylor RD, Schellinger KB. The Impact of Enhancing Students’ Social and Emotional Learning: A Meta-Analysis of School-Based Universal Interventions: Social and Emotional Learning. Child Dev. 2011 Jan;82(1):405–32.
Farahmand FK, Grant KE, Polo AJ, Duffy SN. School-Based Mental Health and Behavioral Programs for Low-Income, Urban Youth: A Systematic and Meta-Analytic Review: SCHOOL-BASED MENTAL HEALTH AND BEHAVIORAL PROGRAMS. Clin Psychol Sci Pract. 2011 Dec;18(4):372–90.
Murray NG, Low BJ, Hollis C, Cross AW, Davis SM. Coordinated School Health Programs and Academic Achievement: A Systematic Review of the Literature. J Sch Health. 2007 Nov;77(9):589–600.
Suhrcke M, De Paz Nieves C. The impact of health and health behaviours on educational outcomes in high-income countries a review of the evidence [Internet]. Copenhagen: World Health Organization, Regional Office for Europe; 2011 [cited 2018 Aug 21]. Available from: http://www.euro.who.int/__data/assets/pdf_file/0004/134671/e94805.pdf
May C, Finch T. Implementing, Embedding, and Integrating Practices: An Outline of Normalization Process Theory. Sociology. 2009 Jun;43(3):535–54.
Cassar S, Salmon J, Timperio A, Naylor P-J, van Nassau F, Contardo Ayala AM, et al. Adoption, implementation and sustainability of school-based physical activity and sedentary behaviour interventions in real-world settings: a systematic review. Int J Behav Nutr Phys Act. 2019 Dec;16(1):120.
Yin RK, Quick SS, Bateman PM, Marks EL. Changing urban bureaucracies: how new practices become routinized [Internet]. Santa Monica, CA: RAND Corporation; 1978 [cited 2018 Aug 14]. Report No.: R-2277-NSF. Available from: https://www.rand.org/pubs/reports/R2277.html.
Pluye P, Potvin L, Denis J-L. Making public health programs last: conceptualizing sustainability. Eval Program Plann. 2004 May;27(2):121–33.
Bumbarger B, Perkins D. After randomised trials: issues related to dissemination of evidence‐based interventions. J Child Serv. 2008 Apr 12;3(2):55–64.
Scheirer MA, Dearing JW. An Agenda for Research on the Sustainability of Public Health Programs. Am J Public Health. 2011 Nov;101(11):2059–67.
Whelan J, Love P, Pettman T, Doyle J, Booth S, Smith E, et al. Cochrane Update: Predicting sustainability of intervention effects in public health evidence: identifying key elements to provide guidance. J Public Health. 2014 Jun 1;36(2):347–51.
Patton GC, Bond L, Carlin JB, Thomas L, Butler H, Glover S, et al. Promoting Social Inclusion in Schools: A Group-Randomized Trial of Effects on Student Health Risk Behavior and Well-Being. Am J Public Health. 2006 Sep;96(9):1582–7.
Herlitz L, MacIntyre H, Osborn T, Bonell C. The sustainability of public health interventions in schools: a systematic review. Implement Sci. 2020 Jan 6;15(1):4.
Stirman SW, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci [Internet]. 2012 Dec [cited 2018 Jul 24];7(1). Available from: http://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-7-17
Braithwaite J, Ludlow K, Testa L, Herkes J, Augustsson H, Lamprell G, et al. Built to last? The sustainability of healthcare system improvements, programmes and interventions: a systematic integrative review. BMJ Open. 2020 Jun;10(6):e036453.
Fleiszer AR, Semenic SE, Ritchie JA, Richer M-C, Denis J-L. The sustainability of healthcare innovations: a concept analysis. J Adv Nurs. 2015 Jul;71(7):1484–98.
Lennox L, Maher L, Reed J. Navigating the sustainability landscape: a systematic review of sustainability approaches in healthcare. Implement Sci [Internet]. 2018 Dec [cited 2019 May 17];13(1). Available from: https://implementationscience.biomedcentral.com/articles/10.1186/s13012-017-0707-4
Allen JD, Linnan LA, Emmons KM. Fidelity and its relationship to implementaiton effectiveness, adaptation, and dissemination. In: Coldtiz GA, Proctor EK, editors. Dissemination and implementation research in health: translating science to practice. New York: Oxford University Press; 2012. p. 281–304.
Bellg AJ, Borrelli B, Resnick B, Hecht J, Minicucci DS, Ory M, et al. Enhancing Treatment Fidelity in Health Behavior Change Studies: Best Practices and Recommendations From the NIH Behavior Change Consortium. Health Psychol. 2004;23(5):443–51.
Moore JE, Mascarenhas A, Bain J, Straus SE. Developing a comprehensive definition of sustainability. Implement Sci. 2017 Dec;12(1):110.
Elias M. Sustainability of Social-Emotional Learning and related programs: Lessons from a field study. Int J Emot Educ. 2010;2(1):17.
Hawe P, Shiell A, Riley T. Complex interventions: how “out of control” can a randomised controlled trial be? BMJ. 2004 Jun 26;328(7455):1561–3.
Shoesmith A, Hall A, Wolfenden L, Shelton RC, Powell BJ, Brown H, et al. Barriers and facilitators influencing the sustainment of health behaviour interventions in schools and childcare services: a systematic review. Implement Sci. 2021 Jun 12;16(1):62.
May C. Towards a general theory of implementation. Implement Sci [Internet]. 2013 Dec [cited 2019 Feb 14];8(1). Available from: http://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-8-18
Bonell C, Allen E, Warren E, McGowan J, Bevilacqua L, Jamal F, et al. Effects of the Learning Together intervention on bullying and aggression in English secondary schools (INCLUSIVE): a cluster randomised controlled trial. The Lancet. 2018 Dec;392(10163):2452–64.
Bonell C, Allen E, Christie D, Elbourne D, Fletcher A, Grieve R, et al. Initiating change locally in bullying and aggression through the school environment (INCLUSIVE): study protocol for a cluster randomised controlled trial. Trials [Internet]. 2014 Dec [cited 2018 Aug 9];15(1). Available from: http://trialsjournal.biomedcentral.com/articles/10.1186/1745-6215-15-381
Bonell C, Allen E, Warren E, McGowan J, Bevilacqua L, Jamal F, et al. Modifying the secondary school environment to reduce bullying and aggression: the INCLUSIVE cluster RCT. Public Health Res. 2019 Oct;7(18):1–164.
Miles MB, Huberman AM. Qualitative data analysis: an expanded sourcebook. 2nd ed. Thousand Oaks: Sage Publications; 1994. 338 p.
Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006 Jan;3(2):77–101.
Pope C. Qualitative research in health care: Analysing qualitative data. BMJ. 2000 Jan 8;320(7227):114–6.
Herlitz L. Understanding the sustainability of school interventions to improve students’ health, wellbeing and behaviour [Internet] [doctoral]. London School of Hygiene & Tropical Medicine; 2021 [cited 2021 Sep 7]. Available from: https://researchonline.lshtm.ac.uk/id/eprint/4659904/
Green LW. Comment: Is Institutionalization the Proper Goal of Grantmaking? Am J Health Promot. 1989 Mar;3(4):44–44.
Shediac-Rizkallah MC, Bone LR. Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy. Health Educ Res. 1998 Mar;13(1):87–108.
Weiss H, Coffman J, Bohan-Baker M. Evaluation’s role in supporting initiative sustainability [Internet]. Cambridge, MA: Harvard University Graduate School of Education; 2002 [cited 2019 Sep 29]. Available from: https://archive.globalfrp.org/publications-resources/browse-our-publications/evaluation-s-role-in-supporting-initiative-sustainability
Elliott DS, Mihalic S. Issues in Disseminating and Replicating Effective Prevention Programs. Prev Sci. 2004 Mar;5(1):47–53.
Mihalic S. The importance of implementation fidelity. Emot Behav Disord Youth. 2004;4:83–105.
Cook CR, Lyon AR, Locke J, Waltz T, Powell BJ. Adapting a Compilation of Implementation Strategies to Advance School-Based Implementation Research and Practice. Prev Sci. 2019 Aug;20(6):914–35.

Download PDF

Version 1

posted

You are reading this latest preprint version

The Sustainment and Adaptation of an English School Intervention to Improve Behaviour, Health and Wellbeing: a Qualitative Longitudinal Case Study

Status:

Version 1

Abstract

Figures

Contributions To The Literature

Background

Method

Description of the case of ‘Learning Together’

Data collection

Trial data (Years 1 to 3)

One-year post-trial (Year 4)

Two-years post-trial (Year 5)

Data analysis

Results

The perceived effectiveness of RP; its sustainment and adaptation

The perceived effectiveness of the actions groups and local actions: discontinuation, sustainment and adaptation

The perceived effectiveness of the SEL curriculum: its adaptation and discontinuation

Discussion

Conclusion

List Of Abbreviations

Declarations

References

Supplementary Files

Status:

Version 1