The vocational school setting
Vocational education in Denmark prepares students for specific occupations and are divided into four main educational tracks: 1) technical (e.g. electrician), 2) business (e.g. office assistant), 3) agriculture and food service (e.g. farmer or chef), and 4) social and health service (e.g. healthcare assistant). Vocational education alternates between school-based training and workplace-based training, with approximately one-third of the time spent in school [36]. Some schools are multi-sited and have student populations greater than 6,000, whereas other schools are located at a single address with a student population of no more than 100. Almost two-third of students are males (64.7%), but the gender distribution varies by educational track; for example, the proportion of males are 89.3% in technical educations, but only 12.3% in social and health service educations [37]. The average age of students is 24 years; some students enrol directly from primary school (15–17 years old), while others begin later in adult life (39.8% > 25 years old) [37].
Recruitment
Eight vocational schools from within four municipalities will be recruited for the study. Schools will be selected based on region (Capital Region of Denmark and Region Zealand) and educational track (technical, business, agriculture and food service, and social and health service) in order to include one vocational school from each educational track in both regions. Simultaneously. the municipalities where the vocational schools are geographically located will be recruited. The enrolled schools and municipalities will be responsible for requiting the community actors, with support from the research team if needed. In this case, ‘community actors’ refer to individuals, organisations or companies – both local and national – who can provide resources and/or social support for implementation of actions for change.
Development and content of the Data Health programme
The Data Health programme offers an iterative five-step process (Fig. 1) for developing change in the systems to shape vocational students’ health behaviour and wellbeing, combining best practices in health promotion research, including methods from systems science [10, 38], data-driven approaches [16, 39] and WHO’s HPS principles [23]. The programme was piloted and formatively evaluated in one vocational school and one municipality from June 2021 to June 2022. The formative evaluation included observations and interviews and surveys with key participants from both school and municipality. Weekly meetings were held between the evaluator (A.S.) and the research team to discuss experiences, barriers and facilitators of implementation. We adapted the specific programme steps accordingly, based on the vocational school context and the importance of potential changes to the programme.
The research team will be responsible for the implementation of step 1–3 of the programme, but will concurrently build the capacity, motivation and commitment of the school and municipal programme coordinators to be responsible for the implementation of actions and to promote the normalisation of the programme (steps 4–5).
Step 1: Collaboration and set-up
The Data Health programme will begin with establishment of a formal collaboration agreement between management representatives from the vocational school, the municipality and the research team. Both the school and the municipality will appoint one or two staff members as school and municipal programme coordinators respectively. The Data Health study will provide one months of salary for each school programme coordinator (EUR 4,000) to support programme implementation. The roles and responsibilities of the different partners will be clarified and negotiated. Once the collaboration and commitment at management level has been secured, information about the programme will be disseminated by the management to all school staff and student. Dissemination activities are developed by the research team and include information by email and a kick-off meeting for staff, while dissemination for students includes customised posters and flyers distributed in common areas of the school. Based on the learnings from the pilot study, we have strengthened the dissemination strategy to make the programme more visible and easier to understand for staff and students.
Step 2: Data-driven approach
Local survey data on students’ health behaviour and wellbeing will be collected and analysed by the research team. To stimulate motivation and interest, the data will be returned to the school and municipality within few weeks in the form of a local health profile. The local health profile covers modifiable health risk factors among vocational students, e.g. nicotine use, unhealthy eating habits, physical (in)activity and poor mental health [7]. A simplified version of the health profile will be presented by the research team and the municipal programme coordinator at separate meetings for school managers and staff, and for students to share and increase knowledge about health promotion, stimulate discussions and reflections about the results, and promote engagement and motivation for change. Based on the data presentations and discussions, staff and students will be asked to complete a short questionnaire to select the health issue they are most motivated to address. During the pilot, we explored how best to select the specific health issue to be addressed in the remaining steps of the programme and who should make this decision. The conclusion was that it is a school management decision, but schools are strongly encouraged to include the perspectives of both staff, students, the municipality and data from the local health profile in the process. To increase the prospect of becoming an iterative model, the local health survey has been designed to match an existing municipal-based health profile system [40] which the school-municipal collaboration can sign up for after the research team has withdrawn the programme.
Step 3: Group Model Building and co-creation of actions
Group Model Building (GMB) is a participatory method from systems science to facilitate a shared understanding of the structures and relationships that shape the system through the creation of causal loop diagrams (CLDs) [38, 41]. A CLD helps to build a shared understanding among participants of the cause and effect relationships within a given system, and to identify and agree on relevant and important areas for change [41]. In previous community research, this approach has been shown to improve understanding of the problem, develop consensus on actions for change among a diverse group of participants, and increase participants’ social network [41, 42].
GMB is a central part of the Data Health programme as we aim to involve and engage a wide range of participants in developing a CLD and identifying locally adapted actions for change. In this programme, the GMB process consists of three sessions (GMB1-3), as proposed and applied by an Australian research group at the Institute for Health Transformation, Deakin University [43]. The Data Health research team was trained in the GMB process by this group and then adapted the process and methods to a vocational school setting and vocational students, primarily based on the approaches and results of the pilot test.
In GMB1-2, participants will co-create a CLD [44] to gain a shared understanding of the perceived causes and drivers of the health issue selected in step 2, e.g. poor mental health or physical inactivity. GMB1-2 will involve 5–10 students, 5–10 school staff, the school and municipal programme coordinators and the school management. The Systems Thinking in Community Knowledge Exchange (STICKE) computer software [45] will be used during this process. At the end of GMB2, participants will identify a number of community actors who are considered relevant or necessary to involve in the identification and implementation of actions across the school, community or municipality. These community actors - representing different organisations/sectors and areas of expertise - will then be invited to GMB3. Together with the participants from GMB1-2 and other interested students and staff members, this wider group of participants will identify leverage points in the CLD and prioritise actions to change the system. Within systems thinking, actions can range from minor actions to major actions [27, 41]. Minor actions often aim to solve a single issue and can often be implemented quickly with low resource costs (e.g. like single events), whereas major actions can aim to change paradigms in the way individuals or organisations think and behave and are often more difficult and costly to implement (e.g. changes in an organisation’s goals and beliefs) [27]. The principle in systems thinking is that minor actions can stimulate major actions [27, 41].
Participants will identify existing actions or co-develop new actions to change specific elements of the system or the whole system. As a result of GMB3, ‘action groups’ of community actors, staff and students will be formed to plan and implement one or more specific actions.
Step 4: Implementation of actions at different levels
In the months following GMB3, the action groups will plan and implement actions for changing the system, assisted by the school and municipal programme coordinators. To support the coordinators and action groups, a guide to the development and implementation process has been adapted from previous work [38]. In addition, the Data Health study has reserved funds for each school (EUR 1,350) to support implementation of actions or involvement of community actors to run implementation.
Step 5: Normalisation and diffusion
In step 5, the research team will organise meetings between relevant community actors and managers from the school and municipality to encourage the initiated collaborations to remain formalised beyond the programme. At these meetings we will develop a strategy for the continuation of the actions already initiated and a plan for new data collection and development of new local actions. Most municipalities in Denmark have a questionnaire tool [40] for data collection to support the municipalities’ work with health behaviour and wellbeing among children and youth. The municipalities will be encouraged to use the tool for future data collection at the programme school as well as at other vocational schools within the municipality.
To increase and maintain momentum for health promotion practices among vocational students, all school and municipal programme coordinators and relevant community actors will connect across study sites every four months in a community of practice (CoP), initiated by the research team. The CoP is a forum to share experiences and learnings on effective strategies for health promotion and collaboration across study sites but will eventually be open to other schools and municipalities interested in implementing the Data Health programme.
If all five steps are completed as intended, we expect various elements of the Data Health programme to be sustained, including the actions implemented, the collaborations initiated, and the organisational practices and motivation to repeat the monitoring to evaluate changes and develop new actions.
Programme evaluation
Research questions
Five research questions (RQs) will comprise the evaluation of the Data Health programme:
RQ1 (Process evaluation - step 2 and 3 of the programme)
To what extent is the data-driven approach and the Group Model Building process implemented as intended, and what seem to be the most important mechanisms of change and contextual factors?
RQ2 (Implementation of actions – step 4 of the programme)
What characterises the planned and initiated actions, and what are the unintended consequences related to implementation? Who is involved in the planning and implementation?
RQ3 (Programme normalisation – step 1 and 5 of the programme)
What are the opportunities, barriers and needs for the collaboratives and programme to be sustained and normalised?
RQ4 (Organisational outcomes)
Does the programme stimulate organisational changes in the schools to work in a more health-promoting direction?
RQ5 (Individual outcomes)
Does the programme contribute to improvement in health behaviours and wellbeing among the vocational students?
Study design
To evaluate the implementation of the programme, related actions and systems impact (RQ1-3), we will use a systems-based evaluation design that seeks to understand how the programme and the systems adapt to each other [10, 46]. Evaluation using a systems perspective needs to adapt as the programme unfolds by examining emergent outcomes that result from the interactions of participants [10, 46], and it involves examining relationships, interactions and patterns rather than individual outcomes and static ‘snapshots’ [46, 47]. The systems-based evaluation design aims to gain an in-depth understanding of the system as a whole, without a ‘control system’. However, it is not possible to include every part of a dynamic system and therefore it is necessary to define systems boundaries to determine what is considered relevant in terms of what and where to evaluate [46, 48]. The starting point for defining the boundaries is the identified primary health issue that each of the eight programme schools has chosen to target. All organisations and interventions relevant to the targeted problem will be considered as part of the system. In terms of system factors, we will focus on modifiable social and physical environmental factors rather than psychological or genetic factors.
We will examine preliminary effectiveness (RQ4-5) at organisational-level (i.e. school organisational readiness) and at individual-level (i.e. students’ health behaviour and wellbeing) using a quasi-experimental design with a non-randomised clustered stepped-wedge strategy. The eight programme schools will be enrolled in two steps, matched by educational track (one school from each main track) and geographical location (two schools from two Danish regions). The matched clusters will be assigned to early start (January 2022) or late start (6 months later). This design allows for a sequential roll-out of the programme and allows us to control for differences between study sites (vertical control) and secular trends (horizontal control) during the study period [49]. Organisational and individual level data will be collected simultaneously at all eight schools four times during the two-year study period at baseline and three follow-up points (T1, T2, T3) (Fig. 2). As systems change take time to diffuse into individual behavioural changes [31], the research team will in due course seek opportunities and funding for longer term data collection (i.e. a five and ten year follow-up).
Programme theory
A programme theory explains how a programme is expected to work and under what conditions [10]. The Data Health programme theory (see Additional file 1) is captured in a series of ‘if-then’ statements and is based on key findings from the pilot test, published literature reporting trail results or theoretical abstractions [8, 16, 27, 50–53], and the research group’s experiences and previous research in the field [20, 54–57]. Mechanisms of change are the hypothesised causal links between the programme components and identified outcomes, triggered within the contexts in which the programme is implemented [10]. In total, we hypothesise six mechanisms of change as plausible causal links between the data-driven approach and the GMB process and outcomes; these are indicated by mechanisms connected to step 2 and 3 in Fig. 3. To understand the interactions between the context, programme components, mechanisms and outcomes; the research group visualised the initial programme theory as a systems map, illustrated in Fig. 3. The map summarises our proposed model of how the programme components are expected to lead to systems change and outcomes. The connecting arrows show how changes in one part of the system are expected to generate changes in other parts of the system. The Data Health programme theory and the system map will continually be refined and revised as part of the evaluation process.
Process evaluation (RQ1)
In the process evaluation [58], we will examine whether the data-driven approach and the GMB process (step 2 and 3 of the programme) are being implemented as intended, the associated contextual factors and the mechanisms of change.
Both quantitative and qualitative methods will be used to collect data for the process evaluation (Table 1). Following the Medical Research Council guidelines (MRC) [58], five process evaluation components will be assessed: recruitment, reach, fidelity, dose delivered and dose received. In addition, we will examine whether the six hypothesised mechanisms of change are activated and the extent to which these mechanisms are modified through their interaction with contextual factors. However, we will remain open to other emerging mechanisms.
Qualitative and quantitative data will be interpreted separately and then combined. Quantitative data will be analysed descriptively, whereas qualitative interviews will be recorded, transcribed and thematically analysed in NVivo [59].
Implementation of actions (RQ2)
Actions initiated by the school, municipality or community actors and the consequences of these actions are a direct indicator of systems change. The number of actions initiated and the primary factors in the locally developed CLD that are influenced will be tracked and visually added to the CLD. The Action Scale Model [27] will be used to determine whether the actions address one of four hierarchical levels to change the functioning of the system in the anticipated direction 1) Beliefs (e.g. the school creates a local action group as health promotion champions), 2) Goals (e.g. the school sets new goals for health promotion), 3) Structures (e.g. school and municipal staff receive training on the complexity of health), and 4) Events (e.g. a single sport event at schools) [27]. The hierarchical structure suggest that changes in beliefs and goals will generate a higher system impact than implementation of single events [27]. We will also examine which participants (i.e. students, school staff and management, municipal staff and community actors) are involved in the planning and implementation of the actions, and how and to what extent they are involved. The involvement levels of participant groups will be interpreted using the involvement matrix [53].
All actions and involvement will be tracked on a quarterly basis up to two years after GMB3 through semi-structured interviews with the school and municipal programme coordinators and involved community actors to enable interpretation of the level of systems change and engagement across participant groups over time (Table 1).
Additionally, an adapted version of ‘Ripple effect mapping’ [60] will be used to understand the rippling effects and potential positive and negative unintended consequences of the actions implemented. The CoP will be involved in a process to explore and visualise unintended consequences.
Programme normalisation (RQ3)
We will explore the opportunities, the barriers and what is needed to integrate the collaboratives and the programme into normalised practice (steps 1 and 5 of the programme).
The school-municipality collaboration and programme normalisation will be explored by interviews with the school and municipal programme coordinators (Table 1). Interview guides inspired by the Partnership Synergy framework [51] will be developed to explore the functioning of the collaborations and the potential for maintenance and the Normalisation Process Theory [61] will be used as a framework for understanding programme normalisation. In addition, carefully written notes will be taken from the programme maintenance meetings, which will be completed as part of step 5 of the programme.
Interviews will be recorded and transcribed, and all data will be analysed thematically in NVivo [59] to understand opportunities, barriers and needs related to sustaining collaborations and normalising the programme. Finally, we will develop recommendations for the implementation of systems approaches to sustainable health promotion in vocational schools.
Table 1
Research questions, items, methods, and instruments/frameworks used in evaluation of implementation (RQ1-3)
Component
|
Items of interest
|
Methods/data
|
Applied evaluation instruments/frameworks
|
Process evaluation - step 2 and 3 of the programme
Research question: To what extent is the data-driven approach and the Group Model Building process implemented as intended, and what seem to be the most important mechanisms of change and contextual factors?
|
Context information
|
For example:
• School type
• School size
• School location
• School and municipal structure
|
Semi-structured interviews with the principal school managers and the municipal programme coordinators (after GMB3)
Facts obtained through school and municipality websites
|
Medical Research
Council guidance [58]
|
Implementation
|
What is delivered:
• Recruitment
• Reach
• Fidelity
• Dose
• Adaptions
|
Registration of participants during meetings and GMB sessions
Exit surveys to all participants (after data presentation meetings, GMB2 and GMB3)
Semi-structured interviews with the school programme coordinators (after GMB2) and the principal managers (after GMB3)
|
Medical Research
Council guidance [58]
COMPACT Stakeholder-driven Community Diffusion Survey [71]
|
Mechanisms of change
|
• Knowledge of health promotion
• Shared understanding of health issues
• Motivation and engagement
• Collaboration
• Shared decision making
• Resource allocation
• Other emerging mechanisms
|
Exit surveys to all participants (after data presentation meetings, GMB2 and GMB3)
Semi-structured interviews with the school programme coordinators (after GMB2) and the principal managers (after GMB3)
|
Medical Research
Council guidance [58]
COMPACT Stakeholder-driven Community Diffusion Survey [71]
|
Implementation of actions – step 4 of the programme
Research question: What characterises the planned and initiated actions, and what are the unintended consequences related to implementation? Who is involved in the planning and implementation?
|
Actions
|
• Number of actions initiated
• Primary variable of influence
• Action level (beliefs, goals structures, events)
|
Semi-structured interviews with the school and municipal programme coordinators quarterly
|
The Systems Thinking in Community Knowledge Exchange (STICKE) computer software [45]
Action Scale Model [27]
|
Involvement
|
• Number of involved participants
• Level of involvement in preparation, execution, and implementation of the developed actions
|
Semi-structured interviews with the school and municipal programme coordinators quarterly
|
The involvement matrix [53]
|
Unintended consequences
|
• Rippling effects and potential positive and negative unintended consequences of the actions implemented
|
The community of practice network will be involved in a process (meetings or workshops) of identifying the rippling effects
|
Ripple effect mapping [59]
|
Programme normalisation – step 1 and 5 of the programme
Research question: What are the opportunities, barriers and needs for the collaboratives and programme to be sustained and normalised?
|
Maintenance of collaboratives
|
• Potentials, barriers and needs
• Partnership synergy
|
Notes from the meetings for programme maintenance (six months after GMB3)
Semi-structured interviews with the school manager (after GMB3) and the municipal programme coordinator managers (six months after GMB3)
|
Partnership Synergy [51]
|
Programme normalisation
|
• Coherence
• Cognitive Participation
• Collective Action
• Reflexive Monitoring
|
Semi-structured interviews with the school manger (after GMB3) and the municipal programme coordinator managers (six months after GMB3)
|
Normalisation Process Theory framework [61]
|
Notes: GMB = Group Model Building |
Organisational outcomes (RQ4)
School organisational readiness to tackle health promotion is selected as a primary organisational outcome. We will apply the ‘Organisational readiness framework’ [50] to track change over time. Organisational readiness refers to the extent an organisation is willing and able to implement change, and the elements in the framework have previously been shown to be related to implementation success [62]. School organisational readiness will be assessed by questionnaires to all school staff, supplemented by semi-structured interviews with the principal managers at the schools (Table 2).
The questionnaire instrument will adapt and refine existing and tested items (e.g. [20, 63]) across three dimensions and ten sub-dimensions: motivation (relative advantage, compatibility, complexity and priority), general capacity (culture, climate and staff capacity) and programme-specific capacity (knowledge, skills and abilities). We will use exploratory and confirmatory factor analysis to initially validate the instrument and assess internal consistency and convergent validity. The questionnaire will be distributed electronically via email to all school staff (e.g. teachers, counsellors and administrators) and repeated data will be collected four times during the study period (Fig. 2). Changes in school organisational readiness will be assessed primarily using linear mixed modelling, adjusting for clustering, with school as a random effect and time as a fixed effect.
Interviews with school leaders will contribute to a deeper understanding of how the support and priorities of the school management can stimulate or hinder the school’s readiness and willingness to change. The interviews will be recorded, transcribed and analysed thematically according to the dimensions and sub-dimensions of organisational readiness.
In order to generate a comprehensive understanding of school organisational readiness, a convergent parallel mixed methods design will be used in the analyses [64]. Quantitative and qualitative data will be analysed independently, then compared, related and interpreted.
Individual outcomes (RQ5)
We will explore indications of effectiveness at the individual level through changes in students’ health behaviour and wellbeing. Items to assess health behaviour will primarily consist of validated measures and items used in other national health profile studies in Denmark (Table 2). The research team will pre-identify one primary and one secondary outcome indicator item for each targeted health areas (i.e. dietary behaviour, physical activity, alcohol consumption, marijuana and drug use, and wellbeing). The primary indicator will be an item assessing students’ health behaviour during school time and the secondary indicator will be an item assessing health behaviour during leisure time or overall.
Following Bauman & Nutbeam (2022) [8], we will assess both health behaviour and health promotion outcomes in order to understand the complexity of health behaviour. Whereas health behaviour outcomes are expressed in terms of changes in, for example, levels of physical activity or mental wellbeing, health promotion outcomes are the personal, social and environmental factors that are a means or mechanism for improving people’s ability to change behaviour [8]. Therefore, health promotion outcomes are considered intermediate to health behaviour (see Table 2 for examples).
Questionnaire data will be collected four times (Fig. 2). Regardless of the specific health issue each school chooses to address, all students will receive the same questionnaire in order to identify potential intended and unintended consequences on other health promotion outcomes or behaviours, as suggested in a complex systems sense [46, 65]. Electronic questionnaires will be distributed by the research team, who will be present in the classrooms so that students can ask questions as they complete the questionnaire. The questionnaire will be completed during school hours and, based on previous experience with this procedure [54], we expect that 90% of the students present during class will complete the questionnaire.
Most analyses will be based on cross-sectional data and descriptive in nature, interpreting mean differences between intervention and control groups. However, analyses will also include linear mixed models with time and selected covariates (e.g. gender, age, ethnicity and socio-economic status) as fixed effects and school as a random effect. It is important to note that the total time of two years to assess individual outcomes may be too short to provide conclusive evidence at this level.
Ethics and dissemination
This study with human participants did not require ethical approval from the Health Research Ethics Committee for the Capital Region of Denmark (journal number: 22012766), as this is not required for social health science in Denmark [66]. The study has been referred to the Capital Region of Denmark’s legal centre for personal data handling (journal-nr.: P-2021-327), allowing collection and handling of personal data. All methods will be carried out in accordance to the General Data Protection Regulation (GDPR) and Danish data protection law [67]. Participation in research related to the study is voluntary and requires written informed consent from all participants. Consent may be withdrawn at any time. Questionnaire data, key files, audio files and transcripts of interviews and observations will be stored in a secure folder on the corporate network in accordance with the requirements of the Capital Region of Denmark and GDPR rules. Only the principal investigator and those with permission from the principal investigator will have access to the data.
The results of this research will be disseminated through national and international conferences, peer-reviewed journals, reports and online. In addition, key findings will be disseminated through the CoP and other national practice network meetings with stakeholder and policy representatives.
Table 2
Research questions, items, methods, and instruments/frameworks used in evaluation of effectiveness (RQ4-5)
Component
|
Items of interest
|
Methods/data
|
Applied evaluation instruments/frameworks
|
System outcomes
Research question: Does the programme stimulate organisational changes in the schools to work in a more health-promoting direction?
|
Organisational Readiness
|
Baseline and change in:
• Organisational motivation
• General capacity
• Health Promotion Capacity
|
Questionnaire to staff members (four times in total)
Semi-structured interviews with principal managers (after GMB3)
|
Organisational readiness [50]
|
Individual outcomes
Research question: Does the programme contribute to improvement in health behaviour and wellbeing among the vocational students?
|
Health behaviour
|
Baseline and changes in:
• Mental health and wellbeing
• Physical activity
• Food intake
• Nicotine use
• Alcohol use
|
Questionnaire to all students (four times in total)
|
WHO-5 [74]
SGPALS [75]
Validated measures from The Danish National Health Survey and The Health Behaviour in School-aged Children Study (HBSC) [76]
|
Health promotion outcomes
|
Baseline and changes in:
• Health Literacy
• Knowledge relevant to the problem of interest
• Self-empowerment
• Self-confidence
• Self-efficacy
• Behavioural intentions
• Motivation
• Social network and interaction
• Social connectedness
• Student autonomy
|
Questionnaire to all students (four times in total)
|
HLQ [77]; HLSAC [78]
And validated measures from The Danish National Health Survey and The Health Behaviour in School-aged Children Study (HBSC) [76]
|
Notes: GMB = Group Model Building |