Design and Setting
The study will employ a 6 month pilot, parallel group RCT design using guidance on undertaking pilot implementation trials outlined by Pearson et al (26) and reporting in accordance to the Consolidated Standards of Reporting Trials (CONSORT) pilot guidance (27). Twenty ECEC services located in New South Wales (NSW), Australia (excluding one region participating in another trial to increase outdoor free play) who report implementing indoor-outdoor free play programs for most of the day since July 2021 (i.e., defined as < 20% of indoor-only free play time during core service operating hours) will be randomised to receive either the SPSH program or usual care. July 2021 was selected as the program initiation date as this was when guidance around outdoor free play programs for ECEC services were released by state education authorities.
Participants and Recruitment
Sampling Frame:
The research team undertook a telephone survey with a randomly selected sample of 231 Long Day Care (LDC) services and preschools in NSW from October 2021 to June 2022, of which 99 services met the eligibility criteria for the current study and form the current sampling frame. LDC services usually operate from 7am to 6pm Monday through Friday and cater for children from birth to school age, while preschools cater for children from 3–5 years and operate from 9am to 3.30pm (28).
Eligibility:
To be eligible to participate, ECEC services must report implementing indoor-outdoor free play programs for most of the day (i.e., offering indoor-only free play for only 20% or less of their core operating hours where most children are present in care. This is can be defined as approximately 84 minutes or less, as standard operating hours are between 8.30am – 3pm) on a typical day since the introduction of the COVID-19 guidance for ECEC services (July 2021) (29). A typical day represents a day where excursions or events which may disrupt the service’s usual routine are not planned. Services will be ineligible if they i) have been implementing indoor-outdoor free play programs prior to the introduction of the COVID-19 guidance (July 2021); (ii) cater exclusively for children with special needs; (iii) have a Nominated Supervisor or staff that do not understand English or (iv) are a Department of Education community-run service (representing less than 5% of services) across NSW.
Recruitment Procedures:
All services in the sampling frame will receive an invitation email consisting of a recruitment package including an information statement, and a link to an online consent form outlining study requirements and requesting participation. Once the consent form has been completed by the Nominated Supervisor, the online survey will automatically generate the recruitment survey to assess eligibility. A member of the research team will telephone services who have not responded after two weeks of receiving the email to discuss study details, confirm eligibility and request consent for study participation. Recruitment will continue until 20 ECEC services have consented. Additionally, the research team will obtain the contact details of an educator who will assist with baseline data collection processes.
Randomisation and blinding:
Following baseline data collection, services will be randomly allocated to either receive the SPSH program or usual care. A statistician will set up block randomisation using a computerised random number generator. Block randomisation will be used to ensure group allocation is approximately equal. This will be uploaded onto the online data capturing platform (REDCap) (30, 31) where service records are stored and concealed from the data collectors. This pilot RCT will be conducted as an open trial as the sustainment strategy cannot be delivered without revealing the allocation to services. Therefore, services and data collectors will not be blinded to group allocation, however, those conducting data analysis will remain blinded to allocation for the main impact outcome (minutes of outdoor free play).
Intervention:
Theoretical Framework:
The development of the 6-month sustainment strategy to increase sustainment of indoor-outdoor free play programs was overseen by an advisory group consisting of implementation and behavioural scientists, health promotion staff and health policymakers, and informed by the Integrated Sustainability Framework developed by Shelton and colleagues (25). The Integrated Sustainability Framework is a comprehensive, theoretically informed sustainability determinants framework developed based on available empirical research of factors identified to influence intervention sustainment across a range of contexts. The framework outlines the dynamic interactions between 21 multi-level factors across five domains including; Outer Contextual Factors, Inner Contextual Factors, Processes, Characteristics of the Interventionists and Population, and Characteristics of the Intervention (25). To identify the determinants related to the sustainment of PA programs, data was collected from a telephone survey with ECEC services nationally assessing the Integrated Sustainability Framework constructs, and extracted from a systematic review of sustainability determinants within ECEC and schools settings (32).
From this data, we identified 20 barriers associated with the sustainment of PA programs in the ECEC and school setting, which mapped to all five domains of the Integrated Sustainability Framework. This resulted in the development of eight strategies (see Table 1) which sought to address these barriers and support the sustainment of implementing indoor-outdoor free play programs. As barriers were predominantly mapped to the Outer Contextual Factors (e.g., Lack of future external funding/financial support, or lack of state requirements) and Processes domains (e.g., Lack of training/professional development opportunities to upskill, or lack of collaboration with community groups), these domains were of particular focus when developing the strategies. The Expert Recommendations for Implementing Change (ERIC) glossary adapted to describe strategies to address sustainment (33) was used to select the strategies, including: identifying an opinion leader to act as a key driver of the sustainment strategy, ii) Affirming the service’s intent for continuity, iii) providing the service with technical support and educational resources to self-direct the sustainment strategy, iv) developing a formal sustainment blueprint, v) engaging with family and community members, vi) reviewing and changing service policy to integrate indoor-outdoor free play programs and vii) developing an ongoing monitoring plan to self-direct the sustainment of the evidence-based practice (EBP) well after the intervention period (34). The intervention mapping process is outlined in Table 1 below.
Table 1
Mapping of sustainment strategies against the Integrated Sustainability Framework and the Action, Actor, Context, Target, Time (AACTT) Framework (25, 69).
Sustainment Strategy | Strategy Description (ERIC strategy and definitions) (33) (33) | Actor(s) / Personnel | Context | Target | Time | Integrated Sustainability Framework Domain + Factor (25) | Barriers addressed (32) | Justification |
---|
Identify opinion leaders at the service | ERIC Strategy: Engage with local opinion leaders. Periodically engage with providers identified by colleagues as opinion leaders or “educationally influential” about the importance of continuing to deliver the practice innovation in the hopes that they will influence colleagues to sustain its use. The identification and engagement with a staff member who could be considered as a key driver or influencer for the original implementation of the EBP. | Health promotion officer/Nominated Supervisor | Phone call | Opinion leader | Prior to the introductory meeting video call | Inner Contextual Factors Organisational leadership and support | Lack of administrative buy-in and support/ leadership/ managementa (70–76) | Identifying an opinion leader aims to target these barriers around leadership/support from management by identifying a staff member at the service that can lead the sustainment strategy to encourage and motivate other staff to continue to deliver indoor-outdoor free play programs. |
Inner Contextual Factors Program champions | Lack of program leaders/ facilitators/ championsa (70, 75) |
Affirm intent of continuity | ERIC Strategy: Affirm formal commitments. Revisit the written commitments obtained from key partners that state what they will do to implement and sustain the innovation. Assess whether these commitments are being upheld and whether new commitments are required to help sustain the innovation. The level of commitment to ongoing implementation will also be ascertained by establishing service motivations for initial and continued implementation, as well as understanding any barriers and facilitators to long-term implementation of the EBP and the sustainment strategies. Developed using these strategies: • Sustainability Action Plan • Motivational statement | Health promotion officer | Video calls/ phone calls | Opinion leader | • Introductory meeting video call • 3-month support meeting • Additional contacts upon request | Characteristics of interventionists and population Implementer characteristics | Lack of motivation/ interesta (47, 73, 76–78) | Affirming the services intent of continuity helps to tackle the lack of motivation barrier by regularly reminding the service of the importance of indoor-outdoor free play programs and its benefits to children. |
Characteristics of the intervention Perceived benefits | • Limited cost-effectiveness/feasibility of programa (73) • Time required to implement and record uptake of the programa (47) • Competing resources, responsibilities and curriculum demandsa (47, 71, 73, 75, 76, 79) | This strategy also targets the barriers of feasibility of the EBP, time required to implement the EBP and competing responsibilities as it focuses on re-iterating the service’s intent of continuing their practice of delivering indoor-outdoor free play programs. This is important as it highlights that the service is already implementing the EBP, thus this strategy of the SPSH program aims to aid the service to continue their current practice rather than change routines or add to staff workload. |
Provide local technical assistance to support integrating strategies | ERIC Strategy: Provide local technical assistance. Develop and use a system to deliver technical assistance focused on implementation and sustainment issues using local personnel. Opinion leader will receive 2 video call support meetings and a minimum of 2 emails throughout the intervention period. The initial video meeting will highlight the purpose and sustainment strategies. Each contact will review progress and provide feedback of integration of sustainment strategies. Additional support will be provided as required. | Health promotion officer | Introductory support meeting: Video call 3-month support meeting: Video call Other intervention contacts (as required): phone/email | NS/Opinion leader | • 1 video call at beginning of intervention • 1 email at the 2-month timepoint • 1 video call at the 3-month timepoint • 1 email at the 4/5-month timepoint | Processesb Training/ supervision/ support | Lack of training/professional development opportunities to upskilla (73–75, 77, 78, 80, 81) | The provision of technical support aims to target this barrier through providing services with the skills/tools needed to continue to implement the EBP themselves without external support. |
Processesb Program evaluation/data | Lack of clear data on effectiveness of programa (47) | In the case of clarity around effectiveness of the EBP, technical support is provided to develop a motivational statement supporting the delivery of the EBP, to ensure services understand the importance of this practice. In addition, this strategy aims to address any challenges experienced when delivering the EBP. |
Develop a formal sustainment blueprint | ERIC Strategy: Develop a formal implementation blueprint. Develop a formal blueprint which includes all goals and strategies to guide the implementation effort over time. This is the introduction of an action plan (“Sustainability Action Plan”) for the service to implement best-practice sustainment strategies. This will track the implementation of the sustainment strategies and their progress. | Health promotion officer and Opinion leader | Introductory support meeting: Video call 3-month support meeting: Video call | Opinion leader | Provided: prior to introductory support call Discussed: during the introductory and 3-month support meeting Action: self-directed throughout the 6-month intervention period | Inner Contextual Factors Organisational leadership/ support | Lack of centralised coordinationa (71) | This aims to address these barriers by developing a Sustainability Action Plan, with defined strategies to support the ongoing implementation of the EBP long-term. The action plan is intended to be a self-directed tool that services use to track their progress of implementing the sustainment strategies, for which the opinion leader is responsible. This strategy allows for more centralised leadership, led by the service, rather than externally. |
Processesb Communications and strategic planning | Absence of plan with defined measuresa (47) |
Distribute educational materials | ERIC Strategy: Distribute educational materials. Distribute educational materials (including guidelines, manuals, and toolkits) in person, by mail, and/or electronically. A resource pack will be provided to services which includes the following resources: 1. Which external organisations support outdoor free play? 2. How outdoor free play aligns with the National Quality Framework 3. COVID-19 outdoor play guidance 4. Australian Physical Activity guidelines 5. Indoor-outdoor free play in any weather fact sheet 6. Benefits of Outdoor Free Play fact sheet 7. Shade and Heat fact sheet | Health promotion officer | Email | Opinion leader | Provided: prior to introductory support meeting Discussed: during introductory support meeting | Outer Contextual Factorsb Funding environment and availability | Lack of future external funding/financial supporta (47, 71, 72, 76, 82, 83) | The resource pack includes a fact sheet on a number of organisations which support ECEC services to deliver and encourage outdoor free play. |
Outer Contextual Factorsb Sociopolitical context | Lack of state requirementsa (78, 84, 85) | The resource pack includes the requirements of outdoor free play in the National Quality Framework and the Early Years Learning Framework which are a part of accreditation standards for ECEC services in Australia. |
Outer Contextual Factorsb Values, needs, priorities | • Lack of evidence-based treatmentsa (71) • Government perceptionsa (47) | The “benefits of outdoor free play” fact sheet highlights the importance of the EBP through an evidence-based lens. Government perceptions are addressed through the fact sheets on the Australian government supporting outdoor free play through the Australian physical activity guidelines, and the NSW COVID-19 outdoor free play guidance for ECEC services. |
Processesb Training/supervision/support | Lack of training/professional development opportunities to upskilla (73–75, 77, 78, 80, 81) | The “indoor-outdoor free play in any weather fact sheet” and the “shade and heat fact sheet” highlight the different ways children can enjoy the outdoors in different seasons while staying safe. This strategy addresses this barrier by exploring the different ways indoor-outdoor free play programs can be implemented at an ECEC service. |
Processesb Program evaluation/data | Lack of clear data on effectiveness of programa (47) | These barriers are addressed through the “benefits of outdoor free play” fact sheet, highlighting the importance of outdoor free play in children due to the many benefits children receive. This EBP is well researched in the literature to be an effective way to increase PA, which in turn has many health benefits for children. |
Engaging with families | ERIC Strategy: Involve patients/consumers and family members. Engage or include patients/consumers and families in the implementation and sustainment efforts. The following resources will be provided to services: 1. Orientation package templates for new families and staff 2. Template for advocating the EBP on the services’ website. 3. Newsletter snippets for families | Health promotion officer/ Opinion leader | Email/Video call | Families /staff at the service Community members | Provided: prior to introductory support meeting Discussed: during introductory support meeting Action: self-directed throughout the 6-month intervention period | Processesb Partnership/ engagement | Lack of collaboration with community groupsa (75) | The use of the newsletter snippets and orientation packages for families allow the service to communicate to families and the community regarding the delivery of indoor-outdoor free play programs at their service. This communication opens up opportunities for families to weigh in and provide feedback as well as advocate to community members the benefits of this practice. By doing so, barriers such as lack of collaboration with community, communicating information, and lack of parental buy-in are addressed. |
Processesb Communications and strategic planning | Communicating information to stakeholdersa (82) |
Characteristics of the interventionists and population Population characteristics | Lack of parental buy-in/support (e.g., uninterested parents)a (47, 73, 75, 76) |
Inner Contextual Factors Organisational stability | Staff turnovera,c (22, 70, 73, 76, 86) | By advocating for the delivery of indoor-outdoor free play programs on the services’ website and the orientation package for new staff, the barrier of staff turnover is addressed as new staff members are aware of the current practices and therefore should continue to uphold these as a new member of the team. |
Review and embedding change into policy | ERIC Strategy: Mandate change. Have leadership declare the priority of the innovation and their determination to have it implemented and sustained. The following resources will be provided to services: 1. Policy template for indoor-outdoor free play programs 2. Policy checklist | Health promotion officer/ Opinion leader | Email | Staff at the service/ Service routine/Service policy | Provided: prior to introductory support meeting Discussed: during the 2-month and 4-month policy review email Action: self-directed throughout the 6-month intervention period | Inner Contextual Factors Organisational leadership/support | Lack of centralised coordinationa (71) | To address these barriers, the provision of a policy template that outlines the benefits of indoor-outdoor free play programs and its consistent provision, enables services to lead the change of their service policy. By allowing services to formally integrate the EBP into their service policy, staff members will have a set of procedures to ensure the provision of indoor-outdoor free play programs as part of their daily routine. |
Processesb Training/ supervision/ support | Lack of training/professional development opportunities to upskilla (73–75, 77, 78, 80, 81) |
Reviewing and embedding the practice of indoor-outdoor free play programs in the service routine as a strategy on the “Sustainability Action Plan”. | Health promotion officer/ Opinion leader | Email | Staff at the service/ Service routine/Service policy | Provided: prior to introductory support meeting Discussed: during the 2-month and 4-month policy review email Action: self-directed throughout the 6-month intervention period | Processesb Communications and strategic planning | Absence of plan with defined measuresa (47) | The “review policy” strategy on the Sustainability Action Plan will allow services the opportunity to review their current service policy and embed the regular practice of providing indoor-outdoor free play programs as part of daily routine. This strategy will provide services with a plan to ensure the provision of indoor-outdoor free play programs long term. |
Develop and implement tools for quality monitoring | ERIC Strategy: Develop and implement tools for quality monitoring. Develop, test, and introduce into quality-monitoring systems the right input—the appropriate language, protocols, algorithms, standards, and measures (of processes, patient/consumer outcomes, and implementation outcomes) that are often specific to the innovation being implemented and sustained. At the initial contact, an “ongoing monitoring plan” item will be added to the Sustainability Action Plan. | Health promotion officer/ Opinion leader | Email/Video call | Staff at the service | Provided: prior to introductory support meeting Discussed: during introductory support meeting and 3-month support meeting Action: self-directed throughout the 6-month intervention period | Processesb Program evaluation/data | • Accuracy in assessing impact of programa (47, 82) • Lack of clear data on effectiveness of programa (47) | The use of an ongoing monitoring tool, such as the FPR, will enable services to monitor their ongoing delivery of the EBP. This tool will highlight the gaps in indoor-outdoor free play provision and address the barriers of lack of professional development opportunities to upskill. By doing so, staff have the opportunity to self-monitor and adjust their practices. By using an ongoing monitoring tool, services will be able to accurately assess the impact of the program, through child benefits, family feedback or other measures, thus enabling the identification of the effectiveness of indoor-outdoor free play programs. |
Processesb Communications and strategic planning | Absence of plan with defined measuresa (47) |
a Barriers identified from interventions in schools (32) |
b Barriers identified from the NSW CATI (see intervention components) |
c Barriers identified from interventions in ECEC settings(32) |
*Table 1 to appear here during production of manuscript*
Sample size
As this is a pilot study, a formal sample size calculation on efficacy was not performed as consistent with best practice guidance (27). However, the researchers sought to recruit 20 ECEC services as this number was considered feasible in the timeframe, and sufficient to provide an adequate indication of the feasibility and acceptability of study methods and the sustainment strategy.
Intervention components
Following baseline data collection and randomisation, services allocated to the intervention group will receive the sustainment strategy. Specifically, services will be supported to continue to implement indoor-outdoor free play programs.
The delivery of the SPSH program will include a minimum of four remotely delivered scheduled contacts, including two meetings via video calls and two support emails (see Table 2). The use of video calls has been found to be an effective and acceptable modality of delivering behaviour change interventions for this setting as an alternative to face-to-face contact (35, 36). Contacts will target the Nominated Supervisor and an educator responsible for supporting the implementation of PA programs in the ECEC service. The intervention will be delivered by experienced health promotion officers (HPO) employed by a local health promotion unit with tertiary qualifications in health promotion and nutrition and dietetics.
Table 2
Month | Type of contact | When | Method | Resources |
---|
0 | Admin | Recruitment | Email | Recruitment Pack |
0 | Admin | Recruitment | Call | Nominated Supervisor Recruitment Survey |
0 | Admin | Recruitment | Email | Free Play Record |
0 | Admin | Recruitment | Email | • Allocation email • Opinion Leader position description |
1 | Admin | Baseline | Call | Admin contact to book 1st intervention contact and identify Opinion Leader |
1 | 1st Intervention contact | 1-month timepoint Introductory meeting | Videoconference call | • Introductory meeting script • Resource Pack • Sustainability Action Plan • Guidance for addressing Barriers |
1 | Admin | 1-month timepoint After Introductory support meeting | Email | Introductory support call summary email |
1 | 2nd intervention contact | 2-month timepoint Policy Review #1 | Email | • Service Policy • Policy Template • Policy checklist • Policy Feedback • Policy Review email |
3 | Admin | 3-month timepoint Reminder email | Email | 3-month timepoint reminder email |
3 | 3rd intervention contact | 3-month timepoint Support meeting | Videoconference call | • Guidance for addressing Barriers • Sustainability Action Plan |
3 | Admin | 3-month timepoint After Support meeting | Email | 3-month support call summary email |
4–5 | 4th intervention contact | 4/5-month timepoint Policy Review #2 | Email | • Service Policy • Policy Template • Policy checklist • Policy Feedback • Policy Review email |
| Additional contacts | As needed throughout the intervention period | Email/Call/Videoconference call | As needed |
6 | Follow-up | 6-month post-baseline timepoint End of intervention | Email/Call | • Free Play Record • Sustainability Action Plan |
| | As needed throughout the intervention period | | As needed |
12 | Follow up | 12-month timepoint | Email/Call | • Free Play Record • Sustainability Action Plan |
a) Identify opinion leaders at the service:
ERIC Strategy
Engage with local opinion leaders (i.e., supervisors or room leaders)
Barriers Targeted
Lack of administrative buy-in and support/leadership/management and lack of program leaders/facilitators/champions
Engaging with opinion leaders is an important strategy to understand the existing internal support for the EBP within the service and is a key driver of implementation in ECEC settings (37–40). ECEC services will be asked to identify any colleagues who they believe are drivers of implementation or “educationally influential”. This may be the Nominated Supervisor or another existing staff member e.g., Room Leader. If this is not the Nominated Supervisor, this person(s) will be invited to participate in service contacts along with the Nominated Supervisor/service manager. The HPO will determine the existing roles and influence of these staff members in current implementation processes as a component of the first 30–60-minute introductory meeting video call contact. The SPSH program will be discussed and additional roles or tasks for these key staff members will be co-developed as part of the process of undertaking a “Sustainability Action Plan” (see strategy d for more detail).
b) Affirm intent of continuity:
ERIC Strategy
Affirm formal commitments
Barriers targeted
Lack of motivation/interest, limited cost-effectiveness/feasibility of program, time required to implement and record uptake of the program, and competing resources, responsibilities and curriculum demands
This strategy aims to assess whether commitments are being upheld and whether new commitments are required to help sustain the intervention (33). Services will be supported to affirm the service’s intent of continuity and readiness for long term implementation of the EBP. To do this, the HPO will explicitly ask the Nominated Supervisor and the opinion leader regarding their intention to continue the implementation of their current indoor-outdoor free play programs during the introductory meeting video call and 3-month support meeting video call. To explore the likelihood of being able to sustain long term change, the HPO will also ascertain the service’s motivations behind their original implementation, determine any barriers and facilitators to ongoing EBP implementation, and readiness to implement the SPSH program to enhance integration into usual care. As part of supporting a commitment to long-term change, the HPO will facilitate services to draft ideas for a formal “motivational statement” for the implementation of the EBP. Services will be supported to use this as part of the staff and community communications, as well as for inclusion in policy and procedure updates as appropriate.
c) Provide local technical assistance to support integrating strategies:
ERIC Strategy
Provide local technical assistance
Barriers targeted
Lack of training/professional development opportunities to upskill, and lack of clear data on effectiveness of program
This strategy has been previously found to support the implementation of an intervention, while also shown to be effective in improving implementation, staff motivation and problem solving within ECEC-based interventions (41–43). HPOs will provide local technical assistance in the form of two, 30–60-minute video calls from the HPO and a minimum of two action-oriented emails throughout the intervention period to assist them with implementing the SPSH program. Video calls will occur at months one and three, with policy reviews via email sent out at months two and four-five. Each contact will draw on continuous quality improvement principles to review progress on the “Sustainability Action Plan” and provide feedback on the integration of sustainment strategies. The first video call will occur after recruitment and will focus on introducing the SPSH program, explaining the roles of the opinion leader and the Nominated Supervisor (see a), understanding any determinants of sustainability for the service (see b), and discussing the development of the “Sustainability Action Plan” (see d). The second video call will be at the 3-month time point which will focus on the service’s progress with their “Sustainability Action Plan” and any experienced challenges. The two emails will consist of providing feedback on the service’s policy, through the use of a template and suggestions of amendments (see g). Additional support contacts may occur at time points outside of those specified if further support is required and will be recorded as part of adaptations to the SPSH program.
d) Develop a formal sustainment blueprint:
ERIC Strategy
Develop a formal implementation blueprint
Barriers targeted
Lack of centralised coordination, and absence of plan with defined measures
Similar strategies haves been used in previous implementation interventions to facilitate improvement in implementation within ECEC services (44). Services will be supported to develop an action plan (blueprint) for the service to implement the SPSH program. This will support the identification of strategies to be put in place as well as sustainment strategies that may have already been undertaken (fully or partially). Examples of strategies include notifying families, community members and staff of indoor-outdoor free play programs, updating the service policy, and conducting indoor-outdoor free play programs as part of the daily routine. Progress with the plan will be monitored at the 3-month support meeting video call, and adjustments and/or support will be provided to help meet strategies as required.
e) Distribute educational materials to support ongoing implementation:
ERIC Strategy
Distribute educational materials
Barriers targeted
Lack of future external/financial support, lack of state requirements, lack of evidence-based treatments, government perceptions¸ lack of training/professional development opportunities to upskill, and lack of clear data on effectiveness of program
Distributing educational materials to services has been found to be highly acceptable among ECEC service staff and is commonly used as part of multicomponent implementation interventions (44–46). Services will be provided with a resource pack approximately 3 days prior to the introductory meeting video call. It will consist of resources developed or adapted from materials previously used by the research team to increase outdoor free play opportunities. The resources seek to target barriers to sustainment and support implementation of the “Sustainability Action Plan”. Resources include fact sheets of external organisations which support outdoor free play, how outdoor free play aligns with the sector accreditation standards, a summary of the COVID-19 guidelines relating to outdoor play, Australian PA guidelines, family, and service templates to include as part of orientation packages, a service policy template, and tips for maintaining indoor-outdoor free play programs across all seasons. Further resources will be developed and provided, tailored to the service’s specific needs.
f) Engaging with family members:
ERIC Strategy
Involve patients/consumers and family members
Barriers targeted
Lack of collaboration with community groups, communicating information to stakeholders, lack of parental buy-in/support, and staff turnover
Engaging with family members and community is critical to support initial adoption and implementation of programs (47, 48). This strategy will include the provision of resources such as 6 newsletter communication snippets providing information around the benefits of outdoor free play, outdoor play in various seasons, and outdoor play and COVID-19. These are to be used in service family newsletters or other communication channels at the service’s perusal, with a minimum to send out 1 newsletter snippet throughout the 6-month SPSH program. A template for the inclusion of indoor-outdoor free play programs in the family orientation package will also be developed. Services will be asked to distribute these using their usual communication channels.
g) Reviewing and embedding change into policy:
ERIC Strategy
Mandate change
Barriers targeted
Absence of plan with defined measures, lack of centralised coordination and lack of training/professional development opportunities to upskill
Implementing a written policy around PA has been associated with higher levels of PA (49–51). This strategy aims to ensure that the commitment to sustain indoor-outdoor free play programs is reflected in a relevant service policy. At the 2-month and 4–5-month timepoint contact, the HPO will review the service’s policy and suggest potential changes to promote the continuation of indoor-outdoor free play programs.
h) Develop an ongoing monitoring strategy:
ERIC Strategy
Develop and implement tools for quality monitoring
Barriers targeted
Accuracy in assessing impact of program, lack of clear data on effectiveness of program and absence of plan with defined measures
Ongoing monitoring is important to allow services to continue to determine if they are implementing the program as intended (52). Services will be encouraged to develop their own plan during the intervention contacts with the HPOs to ensure the EBP continues to be implemented long-term. Alternatively, services can continue to use a tool provided by the research team as a way of continuing to document implementation twice a year.
Control group and contamination:
The delivery of intervention components will be under the control of the research team and will not be provided to the control group services during the intervention period. The ECEC services in the control group will receive ‘usual care’. This includes the provision of a generic email with a link to the benefits of outdoor free play provided by a HPO (29).
Data collection and measures
Baseline data was collected in September-December 2022, while follow-up data collection will be at approximately 6-months (primary timepoint as approximately 2 years post COVID-19 guidelines) and 12-months post-baseline. The following data will be collected using either a telephone or online survey undertaken by a member of the research team.
The Nominated Supervisor/service manager (or the opinion leader responsible for implementation of the SPSH program) will be asked to report on the feasibility and acceptability of the SPSH program as well as the factors that influence the ongoing delivery of indoor-outdoor free play programs at their service (Table 3). In addition, the Free Play Record (FPR) (an internal measure developed by the research team to assess service implementation of indoor-outdoor free play programs and number of minutes of outdoor play) will primarily be completed by the person most appropriate/having knowledge of the conduct of free play at the service (such as the opinion leader or Nominated Supervisor/service manager).
Table 3
Feasibility and Acceptability Measure
TFA construct | Generic TFA questionnaire items |
---|
Affective attitude How you felt about the program | A1. Did you like or dislike the Sustaining Play, Sustaining Health program? Strongly dislike | Dislike | No opinion | Like | Strongly like |
---|
1 | 2 | 3 | 4 | 5 |
|
Burden The amount of effort required to participate in the program | A2. How much effort did it take to engage with the Sustaining Play, Sustaining Health program? No effort at all | A little effort | No opinion | A lot of effort | Huge effort |
---|
1 | 2 | 3 | 4 | 5 |
|
Perceived effectiveness The extent to which the program is perceived to have achieved its objective | A3. The Sustaining Play, Sustaining Health program has helped to continue the sustainment of indoor-outdoor free play programs. Strongly disagree | Disagree | No opinion | Agree | Strongly agree |
---|
1 | 2 | 3 | 4 | 5 |
|
Intervention coherence The extent to which the participant understands how the program works | A4. It is clear to me how the Sustaining Play, Sustaining Health program has helped to continue the sustainment of indoor-outdoor free play programs at my service. Strongly disagree | Disagree | No opinion | Agree | Strongly agree |
---|
1 | 2 | 3 | 4 | 5 |
*Please tell us more about your views |
Self -efficacy A participant’s confidence that they can perform behaviour(s) required to participate in the program | A5. How confident did you feel about participating in the Sustaining Play, Sustaining Health program? [I.e., How confident did you feel about using the sustainment strategy to continue to deliver indoor-outdoor free play programs at your service?] Very unconfident | Unconfident | No opinion | Confident | Very confident |
---|
1 | 2 | 3 | 4 | 5 |
|
Opportunity costs The benefits, profits or values that would have to be given up to engage with the program | A6. Participating in the Sustaining Play, Sustaining Health program interfered with my other priorities [I.e., Did implementing the sustainment strategy interfere with your other priorities?] Strongly disagree | Disagree | No opinion | Agree | Strongly agree |
---|
1 | 2 | 3 | 4 | 5 |
|
General acceptability | A7. How acceptable was the Sustaining Play, Sustaining Health program to you? Completely unacceptable | Unacceptable | No opinion | Acceptable | Completely acceptable |
---|
1 | 2 | 3 | 4 | 5 |
|
A8. Is there anything further you would like to mention regarding how acceptable the Sustaining Play, Sustaining Health program was at your service? |
Feasibility | F1. Continuing to implement the sustainment strategy used in the Sustaining Play, Sustaining Health program seems possible Strongly disagree | Disagree | No opinion | Agree | Strongly agree |
---|
1 | 2 | 3 | 4 | 5 |
|
F2. Continuing to implement the sustainment strategy used in the Sustaining Play, Sustaining Health program seems doable Strongly disagree | Disagree | No opinion | Agree | Strongly agree |
---|
1 | 2 | 3 | 4 | 5 |
|
F3. Continuing to implement the sustainment strategy used in the Sustaining Play, Sustaining Health program seems easy Strongly disagree | Disagree | No opinion | Agree | Strongly agree |
---|
1 | 2 | 3 | 4 | 5 |
|
F4. Is there anything further you would like to mention regarding how feasible the Sustaining Play, Sustaining Health program was at your service? |
Outcomes:
-
Acceptability, feasibility, and potential impact of the SPSH program on the sustainment of indoor-outdoor free play program (defined as ongoing implementation of the EBP) in ECEC services;
Acceptability of the SPSH program:
This is defined as ECEC service staff perceptions of the extent to which the SPSH program is agreeable, palatable, or satisfactory (53).
During the 6-month post-baseline follow-up telephone or online survey, the service manager at the intervention services will be asked to report on 8 items capturing information related to the acceptability (e.g., attitude towards the SPSH program, burden, ethicality of the program, coherence, effectiveness, self-efficacy) of the SPSH program. These items are underpinned by the Theoretical Framework of Acceptability (TFA), a validated framework to assess the acceptability of healthcare interventions from the perspectives of intervention deliverers and recipients (54).
This survey consists of 7 items measured on a Likert scale with response options ranging from 1 to 5. The domains include their affective attitude towards SPSH (“Did you like or dislike SPSH?”), the burden (“How much effort to engage in SPSH?”), the perceived effectiveness of SPSH (“SPSH has helped to continue the sustainment of indoor-outdoor free play programs?”), intervention coherence (“is it clear how SPSH has helped to continue the sustainment of indoor-outdoor free play programs”), self-efficacy (“How confident did you feel about participating in SPSH?”), opportunity costs (“Participating in SPSH interfered with other priorities”) and general acceptability (“How acceptable was SPSH?”). Generally, 1 is a negative/disagreeing with the statement, while 5 is a positive/agreeing with the statement, except for Burden (domain 2) and Opportunity costs (domain 6), where 1 is a positive response, while 5 is a negative/response. The 8th item is an open-ended question to capture any further comments on the acceptability of the SPSH program.
Feasibility of the SPSH program:
This is defined as ECEC service perceptions of the extent to which the SPSH program can be successfully used or carried out at scale within different organisations or settings (53).
At the 6-month post-baseline follow-up time point, the intervention services will be assessed via a survey using 4 items (3 questions on a Likert scale and 1 open-ended item) adapted from the Feasibility of Intervention measure (55). This questionnaire will capture information related to the feasibility (continuing to implement the SPSH program seems possible, easy, and doable) of the SPSH program and its fit within the service. The Nominated Supervisor/service manager will be asked to rate their level of agreement with each item on a scale of 1 (Strongly disagree) to 5 (Strongly agree), along with an opportunity to provide any further comments on the feasibility of the SPSH program via the open-ended item.
Sustainment of indoor-outdoor free play programs (potential impact):
Sustainment as an outcome is defined as: “The extent the innovation is in place or being delivered over the long-term”(56). The delivery of indoor-outdoor free play programs will be assessed at baseline, 6-months and 12-months post-baseline follow up. In this study, it will be operationalised as the number of minutes of indoor-only free play provided by the services over 5 consecutive days.
Data on the number of minutes that children are provided with indoor-only, indoor-outdoor, and outdoor-only free play of a randomly selected week will be collected via the FPR. The FPR is adapted from existing ECEC measures of outdoor play; Environment and Policy Assessment and Observation – Self-Report (EPAO-SR) and Nutritional and Physical Activity Self-Assessment for Child Care (NAPSACC) (57, 58). The adapted FPR measure has been pilot tested with ECEC services and has been reported to take less than 5 min/day on average and found to be acceptable and easy to complete. The FPR will be provided to the Nominated Supervisor/service manager or Educators of the ECEC service electronically via email by the research team, at baseline and follow ups. The Educators/Nominated Supervisor(s) will be asked to fill out the FPR for each session of free play throughout the day across the selected week (5 consecutive days) and return the completed FPR to the research team. If not returned, a follow-up email or phone call will then be made to the appropriate delegate to follow up on the progress of FPR completion. Upon the research team receiving the completed FPR, a phone call will be made to the ECEC service to obtain any missing data or clarify non-legible responses if required and assess any specific challenges to completing the FPR. The research team have successfully undertaken this with 90 services as part of previous trials (59).
Adoption of SPSH program recommendations:
Adoption is defined as the intention, initial decision, or action to try or employ an innovation or evidence-based practice (60). This will be assessed as how many strategies within the “Sustainability Action Plan” were completed by the 6-month time point, to determine how many sustainment strategies were put into practice as per the intention of the SPSH program.
(ii) Feasibility of trial methods to inform the conduct of a fully powered RCT
Feasibility of the trial methods:
In order to inform a fully powered RCT, and to assess whether the trial methods are feasible and able to be replicated, project records will be used to report consent rate, percentage of missing data from completion of the FPR (internal self-reported measure), comparison of consent/non-consenter characteristics, attrition rate, withdrawal and reasons, and contamination (26, 27).
Fidelity of delivering the SPSH program:
Fidelity is defined as the degree to which the SPSH program was implemented as described or intended in the original protocol (60). Fidelity will be assessed using project records, which will determine the extent that/to which each sustainment strategy was delivered by the HPO. This will be done by the research team documenting this data during recruitment and intervention delivery in a spreadsheet.
(iii) Barriers and facilitators to the sustainment of indoor-outdoor free play programs in ECEC services
Barriers and facilitators of indoor-outdoor free play:
During the baseline and 6-month follow-up telephone/online survey with the Nominated Supervisor/service manager, barriers, and facilitators to sustaining the EBP will be assessed. This will be conducted using an adapted measure of the 29-item scale which measures sustainability determinants across the constructs of the Integrated Sustainability Framework (outer and inner contextual factors, processes, and characteristics of the intervention) developed by the team. This adapted 9-item scale measures the outer contextual factors, and processes which are perceived to be influential to intervention sustainment in ECEC settings. The Nominated Supervisor/service manager will be asked to rate their level of agreement to a number of items on a scale from 1 (completely disagree) to 5 (completely agree).
ECEC service characteristics
Data regarding ECEC service characteristics including service type, age group of children at the service, operating hours, number of children enrolled aged three to six years and number of educators at the service will be collected via a baseline survey of ECEC Nominated Supervisors/service managers and educators. Service location postcode will be used to determine socioeconomic status (SES) and rurality. Services that reside in postcodes ranked in the top 50% of state postcodes based on the 2016 Socio-Economic Indexes for Areas (SEIFA) (61) will be categorised into “higher socioeconomic areas,” whereas those in the lower 50% will be categorised into “lower socioeconomic areas.”.
Cost
Similar to previous analysis (62), the direct cost of each sustainment strategy delivered by the HPOs, including labor (i.e. HPO preparation, administration and delivery of the program) will be calculated. Service delivery costs will be recorded by the HPOs delivering the intervention. Costs in AUD $, 2022/2023 will be calculated by multiplying the time spent (in hours) on each sustainment strategy by the hourly wage rate of HPOs delivering the intervention. The cost for Nominated Supervisor/Educators to receive the SPSH program delivered by the HPOs, as well as time taken to implement the strategies will also be calculated. Similar to previous studies examining the cost of receiving interventions within the ECEC setting (63), costs will be calculated and multiplied by the time spent (in hours) receiving the SPSH program by the estimated hourly wage rate of the Nominated Supervisor/service manager and Educators (64). The time spent will account for the services attendance to all meetings and implementation of the strategies at their service.
Overall data management
Management of trial data will be in accordance with a data management protocol, developed and approved by the nominated advisory group. Data will be stored in accordance with the requirements of the Hunter New England (HNE), University of Newcastle, and Swinburne University of Technology Human Research Ethics Committees (HRECs). All questionnaires and notes will be deidentified and recordings will be stored in a locked HNE Microsoft Teams folder to maintain confidentiality.
Statistical Analysis
All statistical analysis will be undertaken in SAS v9.2 statistical software. The main data collection point is at 6 months post-baseline as this will be approximately 2 years from the initial introduction of the COVID-19 guidelines The 12-month timepoint will be used for assessing whether indoor-outdoor free play programs continue to be delivered following ceasing the active sustainment strategy (the SPSH program).
Acceptability and Feasibility
Descriptive statistics (mean and standard deviation (SD)) will be generated for both acceptability and feasibility at 6-months post-baseline. For acceptability, individual scores from each domain will be pooled (reverse scored where necessary) using the overall mean for each service to provide an overall mean score, where ≤ 2 would be a negative response/the SPSH program is considered unacceptable, and ≥ 4 is a positive response/the SPSH program is considered acceptable. Where there are any missing values, this item will be excluded from the analysis, and the completed items will be pooled to provide an overall mean for the service. For the open-ended question, this is be analysed qualitatively using thematic analysis to identify key themes. For feasibility, the individual scores of each of the three items (“the strategy seems possible”, “seems doable” and “seems easy”) will be pooled using the overall mean for each service to provide an overall score, where ≥ 4 (higher scores) indicate greater feasibility, while ≤ 2 (lower scores) would indicate lower feasibility, consistent with that recommended by Weiner and colleagues (2017) used previously in the Feasibility of Intervention Measure. For the open-ended question, this is be analysed qualitatively using thematic analysis to identify key themes.
Sustainment of indoor-outdoor free play
Separate analyses will be performed at each follow-up time point (6- and 12-months postbaseline) for total minutes of indoor-only free play and total minutes of indoor-outdoor free play separately. This will be analysed using generalised linear models with appropriate distribution and link functions, comparing between intervention and control services at follow up and adjusting for baseline. This will include fixed effects for the treatment group (intervention vs control), the baseline value of the outcome and variables that are prognostic of the outcome (service size). Analyses will follow the intention to treat principle and include all data analysed according to the group the centre was originally randomised to. Missing data will be addressed using multiple imputation, with 50 imputed datasets estimated and estimates combined using Rubin’s rule (65). Multiple imputations will be performed for ECEC services not providing follow up data in accordance with the recommendation by White and colleagues (66).
Adoption of SPSH program recommendations
Adoption will be assessed by counting how many strategies were “complete” of the minimum 4 strategies in the Sustainability Action Plan during the 6-month program for each service. The minimum requirement for services is to complete at least one goal from the “Notify families and community members of indoor-outdoor free play” strategy, “Notify staff of indoor-outdoor free play” strategy, “Update policy” strategy and the “Conduct indoor-outdoor free play as part of daily routine” strategy. The number of completed strategies will be summed to determine how many strategies were completed at each service. A minimum of 4 strategies set as “complete” is required for the service to have adopted the SPSH program recommendations. After all intervention services have had their action plan counted, a mean will be calculated, showing the average number of services who have adopted the SPSH program recommendations.
Feasibility of trial methods
Feasibility of the trial methods will be analysed at baseline and 6-months post-baseline. Consent rate, percentage of missing data from completion of the FPR, comparison of consent/non-consenter characteristics, attrition rate, withdrawal and reasons, and contamination will all be analysed using descriptive statistics.
Fidelity of delivering the SPSH program
Fidelity will be assessed using an internal spreadsheet tracking recruitment and intervention delivery. Each step records time spent, staff involved, and whether or not it has been completed. Analysis will include summing the number of completed steps according to the protocol for each service to determine the degree of implementation to the original protocol.
Barriers and facilitators to the sustainment
Barriers and facilitators to sustainment will be analysed at 6-months post-baseline. The outer and inner contextual factors, processes and characteristics of the intervention will be scored individually. The total scores (i.e., the mean and SD) of each of the four domains will be calculated. The total scores will be calculated by summing all scores for all items within the subdomain and dividing by the total number of responses within the domain. The higher domain scores (≥ 4) will be considered facilitators to sustainment, while the lower domain scores (< 4) will be considered as barriers to sustainment.
Progression criteria
An advisory group will assess suitability for progression to a fully powered trial using data collected around feasibility of trial methods, and outcomes related to potential sustainment, fidelity, and acceptability. Such decisions will be made via majority, from core members of the research team, including a representative from a health service that intends to adopt the sustainment strategy if beneficial. Specifically, the team must deem the intervention and sustainment strategy to be sufficiently acceptable and feasible that it would likely be adopted by approximately 25% of ECEC services that were offered it (i.e. a consent rate of > 25%) (67). Alternatively, the team may deem that this could be reasonably expected with adaptations to the intervention or sustainment approach based on steps previously employed by the research team (68). Further, as recommended, measures of delivery of the SPSH program together with acceptability and feasibility, and barriers and facilitators, will be used to identify opportunities to further strengthen the intervention and refine trial methods prior to a full powered implementation trial.
Trial status
The trial is currently in the intervention delivery stage. It is not anticipated that any events would occur that would warrant discontinuing the trial as the SPSH program will be delivered completely online. Any unforeseen adverse events will be reported to the HNE HREC (primary approval committee), and appropriate action taken to address the event. The trial registration record will be updated with any protocol modifications and any deviations from the original protocol will be reported when publishing trial outcomes.