An evaluation of the feasibility, value and impact of using participatory modelling to inform the development of a regional system dynamics model for youth suicide prevention.

doi:10.21203/rs.3.rs-5313398/v1

Download PDF

Research Article

An evaluation of the feasibility, value and impact of using participatory modelling to inform the development of a regional system dynamics model for youth suicide prevention.

https://doi.org/10.21203/rs.3.rs-5313398/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background

Suicide remains a leading cause of death for young Australians. We need robust analytical tools to help us understand and respond to the dynamic complexity underlying suicide and suicidal behaviour. Participatory systems modelling leverages empirical data together with the experiential knowledge of stakeholders to inform model building, evaluation, and implementation. Efforts to evaluate collaborative deliberative processes such as participatory system modelling remain sparse. The aim of this study was to evaluate the feasibility, value and impact of adopting a participatory systems modelling approach for developing a youth suicide prevention system dynamics model in North-West Melbourne, Australia.

Methods

Three participatory workshops were conducted with an expert stakeholder group including young people with lived and living experience of self-harm/suicidal behaviour, family members, healthcare professionals, and policymakers. Data collection was informed by an evaluation framework, including i) briefing questionnaires, ii) workshop recordings and field notes, iii) evaluation questionnaires. Data from each source were analysed separately, using codebook and reflexive thematic analysis, and synthesised to identify patterns in meaning and broad concepts under the evaluation categories of feasibility, value, and impact.

Results

We found that the feasibility of undertaking an inclusive and accessible participatory systems modelling project was influenced by stakeholder cultural diversity, meaningful involvement of those with lived and living experience, accessibility of technical aspects of modelling, and managing group dynamics. The value of adopting a participatory approach in developing a complex systems model was demonstrated in the collaborative learning that took place between the expert stakeholder group and the research team. Central to this was the experiential knowledge of young people who helped shape our understanding of the contextual drivers of self-harm and suicidal behaviour. Stakeholders’ knowledge of the regional health system informed the structure, logic and assumptions of the systems model.

Conclusions

This is one of very few evaluations of a participatory systems modelling project involving young people with lived and living experience of self-harm/suicidal behaviour and bereaved families. Our study offers novel theoretical and practical insights into the processes adopted to ensure the safe and inclusive involvement of diverse perspectives in developing a youth suicide prevention system dynamics model.

participatory systems modelling

system dynamics modelling

lived experience

suicide

self-harm

young people

Suicide is the leading cause of death for Australians between 15–24 years (1). In 2022, suicide deaths accounted for just over one-third of all deaths in those aged 15–24 (a significant increase from 2001, when suicide deaths accounted for 17% and 24% in those aged 15–17 and 18–24 years, respectively) (1). The economic cost of youth suicide in Australia is estimated at $22 billion a year, including direct (ambulance costs, coronial inquiry, policing costs, and funeral expenses), indirect costs (lost economic productivity) and intangible costs (bereavement costs) (2). Despite increasing investment by the Australian government in suicide prevention initiatives (3), suicide remains an intractable public health problem.

We have previously argued that understanding and responding to the dynamic complexity underlying suicidal behaviour within a public health framework requires a shift in thinking (4). System dynamics modelling (SDM) is a computer-assisted method that helps frame, test, and simulate the causal processes and interactions that underlie complex behaviours or systems, such as suicide, to guide evidence-informed decision-making (5). Using SDM, we can model multiple interacting and interrelated suicide risk and protective factors to understand likely pathways of suicidal behaviour, whilst considering the complexity of health systems including factors such as capacity, access, and workforce supply (6, 7). We can subsequently simulate interventions, programs, or policies of interest to determine what will work, for which groups of young people, over what time horizon and via which potential mechanisms (7, 8). Policy makers, health professionals, service planners and commissioners can use this tool to experiment with different “what-if” scenarios and ascertain which of these scenarios will deliver the best outcomes within the system before implementing decisions in the real world (9).

A unique selling point of systems modelling is that it may be guided by a participatory approach to model building, evaluation, and implementation (10, 11, 12). Participatory systems modelling (PSM) refers to “A purposeful learning process for action that engages the implicit and explicit knowledge of stakeholders to create formalised and shared representations of reality” (13, p. 233). PSM brings together an interdisciplinary group of stakeholders to be actively involved in the development of complex models. Stakeholders are facilitated to work collaboratively to identify and structure the problem, map the causal mechanisms underlying the system, identify policy levers, prioritise, and test policy scenarios and support the real-world implementation of those scenarios (8, 12, 14).

There is a proliferation of approaches to modelling with stakeholders guided by different tools and methods depending on the research questions asked, project needs, feasibility, timelines, and model outputs (13). The challenge is how to systematically document and evaluate collaborative deliberative processes, such as PSM, to capture how knowledge and learning is generated, the potential influence of relational and social dimensions of the decision-making process (12, 15), and those characteristics that differentiate nominal from truly transformative forms of participation (16). In modelling projects where expert stakeholder groups include those whose voices have long been neglected due to perceived safety issues, such as young people with lived and living experience of self-harm or suicidal behaviour (17), the need for systematic evaluation of PSM process and outcomes becomes even more critical. However, despite the increasing popularity of adopting PSM to inform mental health and suicide prevention policy and practice (8, 11, 18), efforts to assess and evaluate PSM in terms of achieving its intended aims remain sparse (12, 19). To date, there are no evaluations of PSM solely focusing on young people with lived and living experience of self-harm or suicidal behaviour and bereaved families.

A novel, comprehensive multi-scale framework guiding the evaluation of PSM has recently been published (18) consisting of four categories (including 28 criteria). These categories are: feasibility (i.e., Is participatory modelling feasible?), value (i.e., What is the value of participatory modelling when building complex systems models?), impact/change (i.e., What changed as a result of the participatory modelling?), and sustainability (i.e., What are the longer-term outcomes of the participatory modelling?). Each of these categories evaluates the impacts of PSM across four levels: individual, group, project, and system. The framework (Supplementary Material 1) has previously been adopted to evaluate the process and outcomes of PSM used to develop a complex youth mental health systems model in the Australian Capital Territory (19). The authors found that PSM enabled the involvement of a diverse group of stakeholders, including those with lived experience of mental ill-health, who worked collaboratively to develop a model that had the potential to support evidence-based policy and planning in youth mental health (19). PSM facilitated better understanding of contextual barriers and facilitators for the implementation of the model findings (19).

Aim

The aim of the present study was to evaluate the PSM process used for developing a system dynamics model to inform which programs and policies across health and community settings have the potential to generate the most significant reduction in rates of suicide and attempted suicide among young people aged 12–25 in North-West Melbourne in Victoria, Australia. The study adopted the Lee et al framework (18), to address the following questions:

1. Is PSM feasible for developing a complex systems model for youth suicide prevention in North-West Melbourne?

a) Is it feasible to recruit all necessary stakeholder perspectives in the PSM process?

b) How were power relationships managed?

c) Did all participants contribute and engage during the PSM process (e.g., was it inclusive, accessible, and transparent)?

2. What is the value of PSM in developing a complex systems model for youth suicide prevention in North-West Melbourne?

a) How did the PSM process add value to developing the systems models?

b) What are the facilitators and barriers to developing systems models through participatory methods?

c) What are the experiences (e.g., benefits and challenges) arising from the application of PSM (e.g., positive outcomes and ability to share personal stories)?

3. What is the impact of PSM on the model development process?

a) How was feedback considered throughout the program to improve the PSM process (including the build of the systems model)

Context

The study is part of a wider programme of work (SEYMOUR: System Dynamics Modelling for Suicide Prevention) funded by the European Commission (17) aiming to inform youth suicide prevention policy and practice in Australia and the UK. In Australia, the project took place in North-West Melbourne, which includes some of Victoria’s most socioeconomically disadvantaged and culturally diverse areas (20).

The North-West Melbourne Primary Health Network (NWMPHN), an independent organisation funded by the Australian Government to coordinate primary health care in the region, was identified as the primary partner agency for the modelling project. The research team worked closely with representatives from the Mental Health and Suicide Prevention unit of the NWMPHN to discuss local priorities, scope the model and its boundaries, and facilitate engagement with local, regional, and state-level expert stakeholders for the participatory workshops. The organisation was in the process of developing a new suicide prevention strategy for the region, and the project’s focus was closely aligned with local needs and priorities.

The timing of the modelling project also coincided with the implementation of the recommendations from The Royal Commission into Victoria's Mental Health System established in 2019 with the aim of reforming Victoria’s mental health and wellbeing system. Developing a government-wide approach to suicide prevention was one of the key recommendations of the Royal Commission (21).

Study design

A description of the study design is provided in the study protocol (17). Three participatory workshops were conducted between August 2022 and December 2022. The research team worked collaboratively with an expert group of stakeholders, including i) young people aged 12–25 years with lived/living experience of self-harm and/or suicidal behaviour; ii) family members/carers of young people; iii) primary care and allied health professionals and youth mental health clinicians; iv) health department policy makers and v) service planners and commissioners. Together, these stakeholders conceptually mapped the causal pathways for the development of suicidal behaviour in young people, and their help-seeking journey through health and non-health settings (e.g., community services) including mental health service pathways. A detailed description of participant recruitment and the participatory workshops is provided in Supplementary Material 2.

Safety procedures

The following procedures were implemented to create a safe environment for the involvement of young people with lived and living of self-harm and suicidal behaviour (17):

- A wellness plan completed collaboratively with the young person and a member of the research team, including potential stressors and strategies for young people to use during the workshops if they felt anxious or upset; contact details of a support person who the research team could contact if the young person wished them to; and details of the young person’s primary clinician in case of an emergency.

- A project-specific risk management strategy (17) with clear processes for managing disclosure of suicide risk and/or safeguarding concerns during the workshops.

- A youth advocate (peer with lived/living experience) attended the workshops to support young people and help them navigate the participatory process.

Data Collection

A range of data collection methods was used to systematically document and evaluate the feasibility, value, and impact of PSM process before, during, and after the workshops. The evaluation plan and data collection methods are presented in Table 1.

[INSERT Table 1 HERE]

Pre-workshop activities

A one-to-one online briefing session took place between the first author (MM) and young people to explore barriers and facilitators to their involvement and measures to ensure their safe and inclusive participation. Young people were asked to complete a short questionnaire asking: 1) What are your expectations or drivers for taking part in this study; 2) Are there challenges or concerns you have about your participation; and if yes, would you like to share these; 3) What would keep you engaged in this study/are there any barriers to your continued engagement.

Model building workshops

The research team kept summary points using post-it notes, butchers paper and a whiteboard to document and evaluate the feasibility, value, and impact of PSM. Workshops were audio recorded and two researchers (DK, MV) kept detailed field notes including: i) descriptions of activities, observations, behaviour, verbal and non-verbal communication patterns, discursive interactions as well as potential conflicts, disagreements, decision-making or collaboration between participants, ii) interpretations of what was observed during the workshop e.g., how different stakeholders interacted (or not); how stakeholders behaved towards each other; and explanations of why this was important in relation to the aim of the workshop, iii) reflections of the researcher’s own thoughts, insights, meanings or concerns they might have as they were conducting the observation; any impact they, as an observer, might have had on the workshop.

The research team conducted three, one-to-one consultations with participants who were not available to attend the workshops but wished to be involved and participants who, following the workshops, wished to share additional information or insights in relation to the model development. MM wrote a reflective diary recording thoughts and experiences from stakeholder engagement activities including notes from online and face-to-face meetings, day-to-day procedures, and personal introspections.

Post workshop evaluation

Following each workshop, the research team emailed each stakeholder a debrief questionnaire to collect feedback on: i) aspects of the workshop that worked well and why; ii) aspects of the workshop that did not work so well and why; iii) how the delivery of the next workshop could be improved; iv) whether there were enough opportunities for stakeholders to meaningfully participate in the workshop (e.g., in sharing experiences); and v) whether any aspects of the workshop were upsetting/distressing, and if yes, what helped/did not help. This last question was specifically addressed to those participants with experiential knowledge of self-harm and suicidal behaviour (i.e. young people and family members/carers).

Data analysis

Data from each source (briefing and debriefing questionnaires and field notes/consultations) were analysed separately in relation to the three research questions of feasibility, value, and impact of PSM.

Field notes kept during the workshops were analysed by MM using thematic analysis following a latent, inductive codebook approach (22). MM listened to each audio recording before familiarising herself through repeated reading with each set of field notes. Line-by-line coding of the field notes taken during workshop 1 led to the development of an initial codebook containing code names and definitions for each code, a quote from the data attached to each code, and the evaluation category (feasibility, value, impact/change) and criteria that the code might map onto. During the coding of data from workshops 2 and 3, the initial codebook was amended through a process of constant revision. No qualitative analysis software was used. Throughout this process, MM met with JW and/or JR to discuss patterns in the data and their meaning, as well as reflect on themes and how these might relate to the categories in the Lee et al (18) evaluation framework. Preliminary themes and codes were presented to the wider research team for discussion and refinement. Field notes from the three workshops were analysed in the same way and included in the final codebook.

Free text from the briefing questionnaires was coded by MM who read through all responses and subsequently identified and extracted key components relevant to the research questions guiding the analysis. These components were grouped by common themes under the categories: challenges or concerns about participation; what would facilitate young people’s continued engagement in the study; and expectations or motivation for participating. The same process was followed in the analysis of the debriefing questionnaires the findings of which are presented under the categories: aspects of the workshop that worked well; aspects of the workshop that did not work well; aspects of the workshops that might have been upsetting or distressing; and suggestions for improving subsequent workshops. Insights from the reflective diary were used to contextualise the understanding of stakeholder engagement during the PSM process including the researcher’s own influence of this context.

Findings across the three data sources were reviewed to explore patterns in meaning and broad concepts that could be clustered around the PSM evaluation categories of feasibility, value, and impact and the specific evaluation questions under each category.

Demographics

Sixteen participants consented to taking part in the workshops, three (two young people and one health professional) of whom dropped out after consent citing competing priorities and moving on to a new job as reasons for their withdrawal. The final expert stakeholder group included thirteen participants. Further demographic information and workshop attendance is presented in Table 2.

Around half (53%) of our expert stakeholders identified as having experiential knowledge of self-harm and/or suicidal behaviour: 30% identified as young people with lived and living experience of self-harm and/or suicidal behaviour and 23% as family members or carers of a young person with lived and living experience.

[INSERT TABLE 2 HERE]

Evaluation findings

1. Pre-workshop activities

The briefing questionnaire was completed by the four young people with lived and living experience of self-harm and/or suicidal behaviour who took part in the workshops. Findings are presented under three categories: i) challenges or concerns about participation, ii) what would facilitate young people’s continued engagement in the study, iii) expectations or drivers for taking part in this study. The findings from the first two categories address the question around the value of PSM highlighting the barriers and facilitators to developing systems models through participatory methods. Findings around young people’s expectations and drivers for taking part do not map onto any evaluation category but do provide important contextual information on motivations for participation.

Challenges or concerns about participation

Young people referred to practical challenges that might impede their ongoing involvement in the project including time constraints and availability due to competing priorities. One young person also mentioned feeling apprehensive about having to participate in a face-to-face meeting following the lifting of COVID-19 restrictions. Feeling nervous among other people and fear of how their anxiety might potentially influence their involvement in the workshops was also a concern. Although young people felt passionate about drawing upon their lived experience to inform the project, one of the concerns raised in their responses was talking about their lived experience in presence of "powerful" people referring to clinicians and commissioners. Concerns were also expressed about the possibility for some stories or experiences shared during the workshop to be triggering for others.

What would facilitate young people’s continued engagement in the study?

Young people offered suggestions for processes that could be implemented to support those with lived experience in preparation for and during the workshops. These included: i) giving young people enough notice about the date and timing of the workshops, ii) sending young people the agenda and broad topics to be discussed in advance to give them time to prepare, and iii) incorporating processes to support young people’s meaningful involvement, including having a youth advocate, creating space and time specifically for them to share their thoughts and insights, acknowledging the role of lived and living experience throughout the workshops and having a designated quiet space for them to take a break, if needed.

Expectations or motivation for taking part in this study

Young people described feeling passionate about mental health advocacy for the purpose of improving the health sector (and ultimately helping others) and identified as a key driver for their involvement alignment between the study aim and their vision “Being part of a bigger purpose and vision such as to help young people, improve services and quality of life”. Being keen to contribute to a study which had the potential to generate useful outputs for their region played an important part in young people’s decision to participate. The focus of informing suicide prevention policy and practice in North-West Melbourne, a region described by young people as an “underserved area….it does not get a lot of attention but there is increased need for youth mental health support” was identified as a driver for their involvement.

2. Model building workshops

Three superordinate themes were identified: i) ways to facilitate effective stakeholder working; ii) Benefits of stakeholder engagement; iii) changes to the model following stakeholder feedback. Each theme is presented in detail below and supported by quotes. The thematic framework is presented in Table 3. Findings from the model building workshops offer important insights into the feasibility, value and impact of PSM.

[INSERT Table 3 HERE]

Ways to facilitate effective stakeholder working

Improving diversity and cultural representation within expert stakeholder group

Participants spoke extensively during the first workshop about the lack of diversity and cultural representation within the expert stakeholder group, referring to certain groups of young people not represented. Given that the region where the project was taking place includes some of Victoria’s most culturally diverse areas, participants highlighted the impact of lack of cultural representation on addressing the needs of specific communities in the region.

“Many people are missing from the table (e.g., Indigenous community). This is a mainstream white space. System is racist and everyone's needs are not represented by this group" (CF1).

Participants identified the project and its aims as highly important and timely due to the increased need for mental health support by young people in their region. However, to ensure that the project had real world relevance, participants offered suggestions to the research team about ways of improving the cultural representation among the expert stakeholder group in future workshops.

“Cultural representation is important. If you cannot bring on the table certain groups of young people, then we can talk about bringing in people who work with young people to make this representation better” (AHM1).

Participants working across health and community settings in the region offered to promote the project among their networks, foster relationships with community leaders and support the research team to engage with a diverse group of young people in the community to ensure their needs were sufficiently represented in the expert stakeholder group.

Facilitative leadership in managing power and social dynamics within stakeholder group

Observations of participants’ behaviour and interactions highlighted the presence of dominating voices in the room. This manifested in attempts to lead or navigate the discussion towards a certain (or preferred) path; taking control of the conversation during small group activities by asking leading questions or diverting focus from the task at hand; referring to one’s position, experience, or expertise to influence conversations; and not allowing other stakeholders to share their views. We provide below an extract from the field notes to illustrate this. During the second workshop, the facilitator reiterated that the focus of the model was on mapping health and community systems and how we could maximise those systems to improve suicide prevention initiatives in the region. One of the participants (SF1), who had not joined the first workshop, disagreed with the focus of the project being on systems and insisted that the team should be focusing on individuals and intrapersonal risk factors. In the extract below, having taken over the conversation and insisting on changing the focus of the mapping exercise, SF1 abruptly interrupts another stakeholder (CF1) in their attempt to bring back the conversation to the original aim.

SF1: We are talking about systems when we should be looking at ourselves talking about the individual.

CF1: But it’s also….that sense of individualism is a very western concept…. [gets interrupted by SF1 before finishing their sentence]

SF1: Yes, now you are talking about individualism vs. collectivism and that is the same thing as in order to heal yourself you will be healing your community.

Observations of power dynamics were reported throughout the three workshops and involved participants across all stakeholder groups. For example, young people who had prior youth advocacy experience appeared more confident in leading group activities than young people with no prior advocacy or research experience. During the intervention mapping exercise (workshop 3), it was noted that two young people had taken over the task of listing services and interventions in the region, conversing between themselves without including other young people at the table. The researcher taking the field notes made the following observation “The remaining young people are not contributing at all to the “discussion” because it is very much (name of young person) listing the list of services, and another young person writing them down. This activity was definitely overpowered by (name of young person) views, and the other young people are not represented in here, and there is some indication that they don’t share all of [their] views”. In another example, a health professional was observed interacting with carers during a group task. The health professional appeared to be leading the discussion, asking one carer many questions including about a family member the carer was caring for, and their experience of mental health services. The extensive discussion of the family member’s case contributed to the carer's discomfort. The carer eventually became upset and left the room for a while. When the carer returned, they told the group that they were going to completely leave the workshop as “it's become too much for me”.

Several observations of the facilitators’ efforts to manage power dynamics within the expert stakeholder group were recorded throughout the workshops. Such attempts included: i) reminding stakeholders of the ground rules, ii) picking up signs when someone was trying to dominate or divert discussion and using this opportunity to re-clarify workshop aims, iii) ensuring all voices were heard by taking stakeholder feedback into consideration when building the systems model, iv) facilitating group discussions using guided questions and prompts, v) ensuring circular flow of knowledge i.e., checking and clarifying stakeholders' understanding, and vi) empowering participation by validating stakeholders’ views. We provide an example supported by relevant quotes below.

The example involves a conversation between a stakeholder (SF1) and one of the facilitators (F1) during the intervention mapping exercise about whether the policies of interest to be tested in the model should focus on prevention vs. intervention. SF1, who participated in the workshop as a family member, was leading the small group task and categorically disagreed with identifying suicide-related policies that would focus both on intervention and prevention. Despite repeated calls by other stakeholders for SF1 to be inclusive of everyone’s views on the table, SF1 became increasingly frustrated by the fact that others did not agree with their views. SF1 referred to their experiential knowledge of having lost a loved one to suicide to highlight the need to focus solely on prevention:

“I don't know how many people in the room have actually lost someone so close to suicide (which is relevant due to the relative amount of grief it causes)….. I think people need to ask themselves: do you want suicides to stop, or just reduce?” (SF1).

One of the facilitators (F1), having picked up signs of irritability in the group and some stakeholders being quiet and withdrawn, approached the group to ask about progress with the task.

SF1: Talking about [name of service] is irrelevant to be honest. We need to be talking about risk factors, what causes someone to develop suicidal ideation in the first place.

F1: We need to be talking about both because there will be people who are becoming increasingly unwell and for whom preventative efforts at that early stage might not work....and they might end up needing specialist help.

SF1: I guess what you are talking about is how much you want to focus on prevention vs. intervention. Is it 50–50?

F1: We will focus on both. As agreed, we will go over the model now and at the second part of the workshop we will talk about interventions you will identify and where the gaps are. So, if you, as a group in your table, want to focus on prevention that is absolutely fine.

In the example above, as a way of managing the dynamics of power between SF1 and other stakeholders, the facilitator i) acknowledges the importance of the stakeholder’s views (i.e., focus on prevention) but also validates the opposite view (i.e., focus on intervention) explaining why there is need for both; ii) repeats and clarifies the aims of the task, iii) explains the process the group needs to follow and iv) emphasises the need to identify policies of interest “as a group”.

Improving inclusivity and accessibility of technical aspects of the modelling

Facilitators had to communicate technical information and use modelling language that stakeholders were unfamiliar with. To ensure that this information was accessible, different methods were used including brief slideshows, charts, tables, and concise summaries reflecting on progress made, checking, and clarifying stakeholders' understanding and highlighting outstanding issues to resolve. Although these methods helped distil the technical aspects of modelling, some participants experienced difficulties in meaningfully engaging with certain activities referring to lack of familiarity with specific terms or difficulty in processing how information was presented. For example, a group-based activity during workshop 3 required stakeholders to define and describe “Peer-led interventions”, as one of the prioritised interventions for testing, by working through a list of questions (Supplementary Material 3). Observations of the group’s interaction extracted from the field notes demonstrates their difficulty in relating to the technical language used in the list provided to them “The definition of the term components was too complex. The group asked for a definition but was confused with the explanation offered by the facilitator. Stakeholders had difficulty to define any component and what exactly they would want from the peer worker. The group could not complete the task and left this question for later. The group also appears very confused with the terms reach and adoption. The instructions need to be simplified and perhaps offer examples”.

In a similar example, a group of stakeholders were discussing the fact that there are young people whose presentation and needs will be too complex for primary care, but not unwell enough for tertiary services including hospital admission or state-run specialist mental health services. One stakeholder referred to the term “secondary services” as a way of bridging the divide between primary and tertiary services in the region. Everyone in the table appeared to be familiar with the term “secondary services” and engaged in conversation about what this would look like for the region of North-West Melbourne, except for one stakeholder:

“I feel I don't know enough about the service landscape to be able to decide on what secondary services should look like…I am just unfamiliar with the concept”. (CF2).

Observations of this stakeholder’s non-verbal behaviour during this task mention “….[name of stakeholder] was feeling overwhelmed by the exercise and kept saying she doesn’t have the expertise to suggest things. Eventually, [name of stakeholder] asked for a break and left the room”.

The complexity of information and technical concepts of the workshop might have led to some voices being marginalised during the workshops.

Benefits of stakeholder engagement.

The role of lived experience in shaping and informing the participatory modelling approach

Young people drew upon their personal experience of navigating the region’s health system to identify contextual factors, such as availability and accessibility of services, for understanding how the system does (or does not) work. The process of drawing upon personal stories to inform the systems model was identified by stakeholders as invaluable, offering credibility to the participatory approach and overall project.

“A young person's experience or story is different from collecting data at a single point in time with very specific questions. Stories give context to the data and help see how it is different from another person's story/experience. Stories add dimension. Data (that you'd get from a questionnaire or other type of psychometric measure) from two people might be the same, but there are two different stories and contexts”. (AHM1).

Participants spoke extensively about the value of adopting a bottom-up approach to developing a systems model by bringing together an interdisciplinary group of local experts and placing those with lived and living experience front and centre. Validation of young people’s experiential knowledge and transparency about how this knowledge provided context and learning during the model building process helped to close the gap between researchers and the community.

“Researchers will spend a lot of time building an evidence base for treatment effectiveness, but this rationale is not available to the public, therefore there is no trust”. (AHM1).

The concept of trust was often mentioned in participants’ narratives when referring to how the participatory modelling approach helped build trust in the relationship between the research team and the expert stakeholder group; and by doing so, increasing confidence in stakeholders’ involvement in the model building process.

Sharing and exchanging of knowledge

Throughout the workshops stakeholders engaged with each other through dialogue and debate with some activities (e.g., small group discussions) encouraging more active and productive interaction manifested in sharing and exchanging knowledge, validating concerns but also challenging each other. Stakeholders often had different views on how regional services operated or different experiences of delivering and receiving services. It was through the process of dialogue and debate that stakeholders had the opportunity to discuss disagreements, ask questions and find resolution collectively. This process of collaborative learning helped to connect knowledge across the expert stakeholder team and foster in-depth understanding and awareness of the system.

The following extract involves four stakeholders – two health professionals (AHM1, CL_F_2), one commissioner (SP/C_F_1) and one carer (CF2)- debating the pros and cons of introducing a central coordinating system to manage mental health referrals, waitlists, and pathways to care in the region. Whilst three of the stakeholders drew upon their professional and experiential knowledge to argue for and against adding the central coordinating system as a hypothetical intervention to be tested in the model, another stakeholder informed the group that such an initiative (referred to as “client management interface”) already exists in the state of Victoria.

AHM1: There are too many narrowly specialist services so there is a lot of siloing. We need to have a shared waitlist between the different services, some kind of a centralised coordination centre that manages it all.

SP/C_F_1: In the housing services, they have a central coordinating system; they have entry points but also an online data referral pathway where services will advertise what their services and capacity is. For example, in theory you’d have someone present to an access point, they would then be able to look at what available resources are e.g., a crisis bed, and then they hold that client until they are referred into the service; the service then engages with that client; they might support or find what else might be available after they’ve addressed their immediate needs. The challenge would be in this space….there are so many different players within the mental health system that requires coordination. So, I think that would be the greatest challenge.

CF2: We just need to be aware of the pros and cons of centralisation and decentralisation. I worked in the homeless services for 14 years and there are so many problems with it (referring to centralisation of services). If it was working well, it would be great. I also want to talk about relationships between services...e.g., not knowing what [referring to name of service] criteria are...I find that odd. We all know that there are liaison roles, there are roles that are set up to do these things to help the system be more coordinated, but relationships are so important...good quality relationships.

CL_F_2: Are people familiar with the CMI (client management interface)? It’s statewide and you can see what services the person has accessed etc. There’s also a different system to see how many beds there are available statewide.

The sharing of this piece of information by CL_F_2 about the existence of CMI changed the course of

the conversation from developing and modelling the effects of a central coordinating system, as a

hypothetical scenario, to whether existing data could be sourced to support the research team in

evaluating its effectiveness. Additionally, CL_F_2’s input raised awareness among the expert

stakeholder group about the existence of an important resource for the mental health sector

facilitating the sharing of statutory data with the Victorian Department of Health and other services.

Changes to the model following stakeholder feedback

Stakeholder insight informing model structure and logic

Stakeholders provided unique knowledge on how the (mental) health system operates in their region and this knowledge was used to inform, review, and revise both the structure of the computer simulation model and the logic and assumptions underlying the model. The example below illustrates how stakeholder feedback led to the addition of an important source of help-seeking in the model, labelled “Online services”, which the team had not previously considered.

CF1 I've just been thinking about means of suicide and we know that it’s different for young men and women; we know it’s different for First Nations communities; we know it’s different for LGBTQIA + young people who have a harder time. So’ I'm just wondering how we....online services…young people could be looking online for services to seek help and that’s not captured there.

F1: So, perhaps what we need is an arrow from seeking help to online sources of help for example, helplines?

CF1 Yeah, helplines... all sorts of stuff

Stakeholders highlighted causal pathways between variables in the model and helped the research team understand dynamic behaviours that drive some of the vicious cycles identified in the model.

SP/C_F_1: Care in Mind and Head to Health are also available services in our region. However, just because someone “completes” a treatment, doesn’t mean they’re actually recovered and now mentally well. They end up returning to the [mental health] system.

F2: Like an “vicious loop”? So, the times someone goes through the system, the more it adds to their distress?

SP/C_F_1: Yes, and this then makes the system a contributing factor to a young person’s distress….

Identification of interventions and policies of interest for testing

Stakeholders worked together to identify weaknesses in the system that could be used as leverage points where one could intervene to optimise system behaviour. In the example below, one participant raised the issue of long waiting times experienced by young people at different points in their journey (e.g., between referral acceptance and first appointment) causing unnecessary delays in the system at a time when they are most vulnerable.

“There are many points in the system where young people spend a lot of time waiting……not just [when moving] between services but also within the same service; for example, waiting between headspace appointments. Just because you are in a service that doesn’t mean you don’t have to wait for each appointment”. (SP/C_F_1).

Stakeholders reflected on how the system could be improved through different policies, interventions, or service planning decisions. The outcomes of the voting exercise (Workshop 3) helped the team identify the top three interventions prioritised by stakeholders as the most locally appropriate and relevant for inclusion in the model (1):

I. Having a designated mental health professional in GP clinics (Mental health service planning interventions - increasing mental health service capacity)

II. Community mental health outreach program with a focus on peer-led interventions (Mental health interventions)

III. Family interventions (Specific suicide prevention interventions).

3. Post-workshop evaluation

Debrief questionnaires following the first, second and third workshops were completed by 7/9, 7/10, 3/7 participants, respectively. The main themes from the debrief data are presented and supported by relevant quotes; these offer insights relating to the feasibility, value, and impact of PSM .

Workshop 1

Aspects of the workshop that worked well.

♣ Facilities and workshop format including delivery methods

All participants commented positively on the chosen setting, location, and organisation of the workshop. The format of the workshops, including the variety of delivery methods adopted was highlighted as key to accommodating the needs of a diverse group of stakeholders whilst meeting the project objectives. Being flexible about how information was communicated and the different ways in which stakeholders could share their insights (during and after the workshops) fostered inclusivity and meaningful engagement.

“I thought the format was good too…. The discussions, in smaller groups, before we committed to writing some of our ideas down on the sticky notes worked very well, the colour coding was clever and the full group discussion after bringing it all together was also really good. The range of participants also good”. (CF1).

Small group discussions gave participants the opportunity to share their views in a more informal way and have deeper conversations in a safe space whilst building up their confidence to speak in front of the wider group.

“The small group discussions feeding into the larger, facilitated discussions also worked well in terms of feeling listened to and having opportunities to contribute — it’s easier to do so in a small group, and then can provide confidence to speak up in a larger group”. (YP1).

♣ Workshop facilitation

Workshop facilitation was highlighted in participants’ responses as one of the key factors for the success of the first workshop. Examples of facilitative leadership included the provision of a clear explanation of the project aims and process, offering multiple opportunities to stakeholders to be involved in different ways and being respectful, inclusive, and validating of stakeholders’ views.

“I am not a great fan of a lot of powerpoint but I thought you explained the process well and I understood the task”. (CF1).

“The facilitators successfully directed the conversations and allowed targeted but also diverse discussions. I definitely felt listened and had enough opportunities to contribute throughout the day”. (AHM1).

Aspects of the workshop that did not work well.

Participants identified two key issues related to i) the duration of the workshop (full-day) which was perceived as demanding and might have led to the number of participants slowly dropping during the afternoon session; and ii) lack of adequate representation of young people with lived and living experience of self-harm and/or suicidal behaviour.

“The fact that for the majority of the day I was the only young person didn’t really work too well — I think I managed fine, but it really wasn’t ideal”. (YP1).

Suggestions for improvement workshop 2

Participants identified three ways in which the second workshop could be improved. Firstly, there was a clear need to improve diversity and representation within the expert stakeholder group in relation to the involvement of those with lived and living experience.

“I think there is a "human element" missing from the session/s. I would like to suggest that at some time in the day we hear a statement or a share from a young person or someone with lived experience to remember the seriousness of the impacts of everything to do with suicide and what helps in recovery - not to distress participants or bring the group down, but to remember the real human element in our modelling journey”. (CF2).

“It’s generally best practice to have at least 2 young people in any meeting such as this one, especially in an environment with mental health professionals”. (YP1).

Secondly, participants recommended that the duration of future workshops should be shortened to a half-day to prevent attrition and ensure “continuity of contributions and more collaboration across the range of participants throughout the day”. (CF2).

Thirdly, as the development of the conceptual map became more complex, a suggestion was to allow more time for deeper thinking about the bigger picture in relation to improving the mental health systems in the region.

“Only challenging part for me was that figuring out how to deal with the growing complexity of the model building process, as you may remember that the map became very complex by the end of the workshop. I felt like I lost my connection with the big picture a bit, as I focused more on the details. I may just suggest adding a section or allowing more time for attendees to further workshop specifically on how to fit all these detailed pathways in the big picture and encourage them to focus more on the ideas around how these services and pathways could be placed in a way that the map looks a little easier to understand and less overwhelming”. (AHM1).

Aspects of the workshop that might have been upsetting or distressing.

No adverse incidentswere reported during the first workshop. A certain level of apprehension and nervousness was noted by one carer who said: “I was a bit nervous and shy and worried about crying but everyone was lovely and welcoming and seemed to value my contribution” (CF2). Feelings of validation and the sense of having a shared goal was one of the reasons the workshop was described as “hope instilling”.

“Not particularly [referring to feeling upset during the workshop]. I think it was a really great discussion and being surrounded by others looking for solutions to the problem was really hope-instilling”. (YP1).

Workshop 2

Having reviewed the feedback from workshop 1, the research team recruited two more youth stakeholders, shortened the workshop to half a day, and allowed more time for discussion and reflection.

Aspects of the workshop that worked well.

♣ Workshop facilitation

Participants highlighted many examples of good practice including facilitators encouraging participation but not putting pressure on stakeholders to share, supporting stakeholders in unpacking the complexity behind some of the topics discussed, and clearly showing how stakeholders' views and feedback were used to inform the content of the workshop.

“In terms of the aspects of the workshop that worked well for me. Well, it's more or less the fact that at no point was I pressured into speaking or anything like that. The relevant information was presented in a way that would help bridge the gap of not really being as informed as most people there. I was in-fact asked at multiple points if I had anything to share, and as such I feel that I was given ample opportunity to share my views and story.” (YP2)

♣ Workshop format and delivery methods

Participants commented positively on the revised organisation and delivery of the second workshop. These changes, implemented in response to participants’ feedback, facilitated stakeholders’ involvement in a more meaningful way during the second workshop.

“I think the reduced time was better for this second workshop and I like your mix of big picture and detailed small group work. I felt able to contribute”. (CF1).

Aspects of the workshop that did not work well and suggestions for improving workshop 3.

Four out of the seven participants who provided feedback on the second workshop highlighted the presence of “stronger” voices in the room as problematic. Participants referred to dominating voices which overpowered conversations, tried to navigate the discussion away from the aims of the workshop, and repeatedly referred to their experience or expertise as a way of influencing conversations and invalidating or dismissing others’ contributions.

“I’m aware that this isn’t necessarily something you can control, but at times some of the stronger voices in the room seemed to overpower and even invalidate the experiences of other young people, healthcare providers, etc. in the room, which may have made it difficult, particularly for young people, to speak up and voice their opinions on things. In the same vein, some of these views seemed to fall outside the aim of the model and led to the group being sidetracked and not being able to focus completely on the topic at hand.” (YP1).

The impact that this had on group dynamics is evident in the quote below:

“[Participant’s name] seemed to shut down other people mid-sentence if [they] did not agree with what they were saying. It made myself and at least one other person feel uncomfortable. There were times where I didn't share what I wanted to say because I feared I'd be bluntly interrupted and shut down if this [person] did not like what I had to say”. (YP3).

Going forward, participants highlighted the need for a more detailed debrief at the end of the workshop due to the sensitive nature of some of the conversations that took place.

Aspects of the workshop that might have been upsetting or distressing.

Young people and carers reported feeling somewhat upset due to the nature of the topic of suicide and/or suicide bereavement. One young person reported feeling upset by the fact that there were stakeholders who overpowered the conversation and invalidated their lived experience. Despite these concerns, participants reported that things which helped manage their feelings were good facilitation (e.g., managing group dynamics) and, for young people specifically, the use of strategies recorded in their wellness plan.

“There were occasions that I felt upset or distressed, but not to any excessive degree. More that the topic of conversation was sometimes uncomfortable, and as such I tended to zone out on occasion. There isn't as far as we know anything that can be done to reduce this in the future, as the subject of discussion is more or less the cause, and it is more than tolerable at those levels”. (YP2).

Workshop 3

Only three out of seven participants provided feedback following the completion of the third workshop. We provide a narrative summary of participants’ views of things that worked (or did not work) and how the PSM process could be improved in the future. No adverse incidents were reported. Participants highlighted the fact that having a smaller expert stakeholder group worked very well as people were able to have more meaningful conversations and felt confident sharing their views without worrying about being judged. The quality of facilitative leadership was once again highlighted in relation to meaningfully incorporating lived experience and professional knowledge thereby ensuring inclusivity and transparency in the PSM.

“[name of facilitator 1] and [name of facilitator 2] facilitated the workshop very well and was inclusive (and validating) of all feedback and suggestions made. It was obvious that you both were appreciative of the insights provided and would make a concerted to utilise the feedback in a constructive way that reflects the intention of the feedback. Managing a space where you are incorporating lived experience and professional knowledge can always been challenging as both contributions are unique and valid, and at times, can be contrasting – you were able to navigate this space through validation and the through small group discussions”. (SP/C_F_19).

In terms of improving future PSM workshops, participants had two suggestions: i) delivering workshops in the community, as opposed at a mental health service, as this would improve inclusivity and accessibility, ii) ensuring that small groups are mixed and incorporate both lived experience and professional knowledge.

Findings from each data source are summarised and tabulated (Table 4) to demonstrate patterns in meaning and identify broad concepts that clustered around the PSM evaluation categories. Data tabulation ensured that key findings from each data source were not overlooked therefore enhancing trustworthiness in the data analysis process, synthesis and interpretation of conclusions (23).

[INSERT Table 4 HERE]

This study offers new insights into the feasibility, value, and impact of adopting PSM to develop a complex systems model for youth suicide prevention in North-West Melbourne. Drawing upon the extant literature, we present a critical analysis of the study findings which has informed the development of a theoretical framework of the key ingredients for the successful delivery of a PSM project.

Is PSM feasible for developing a complex systems model for youth suicide prevention in North-West Melbourne?

The findings of the evaluation identified several factors that influenced the feasibility of undertaking an inclusive, accessible, and transparent PSM project. These factors included appropriate representation of those with lived and living experience, stakeholder cultural diversity, translation of technical information and effectively managing group dynamics.

Those with experiential knowledge of self-harm or suicidal behaviour were identified as key beneficiaries and were appropriately represented in the expert stakeholder group (53% of the sample identified as having lived and living experience including young people and carers). The meaningful involvement of young people with lived and living experience was facilitated through the appointment of a youth advocate, ensuring that the views of young people were adequately represented and valued. The role of a youth advocate has been previously recommended as a key strategy for facilitating youth involvement in research, ensuring a positive working synergy between researchers and those with experiential knowledge (24).

Although lived and living experience of self-harm and/or suicidal behaviour was appropriately represented in the PSM, cultural diversity was not achieved despite the research team’s efforts². Difficulties in recruiting all necessary stakeholder perspectives in the PSM process may have implications for the cultural appropriateness of the model we developed as this might not accurately reflect the needs and experiences of all young people in the region. Involving culturally diverse communities in participatory youth mental health systems modelling entails adapting participatory processes to acknowledge the presence of cultural differences (25). In our case, for example, hosting the participatory workshops in the community (e.g. youth club), as opposed in a clinical service, could have made it more feasible and accessible for young people from diverse cultural backgrounds to be involved.

In line with previous findings (13,15,19,26,27), effective facilitation was consistently highlighted by stakeholders as key to managing power dynamics, empowering participation, and ensuring transparency in how professional and experiential knowledge were incorporated in the PSM process. Challenges in successfully incorporating the technical aspects of modelling and the experiential knowledge of stakeholders could have inadvertently compromised accessibility thereby exacerbating power dynamics between the research team and stakeholder group (27). The value of clear, transparent, and non-technical communication, as a way of ensuring inclusivity in the PSM process, has been highlighted in a previous PSM evaluation study (19). The lack of optimal management of the technical aspects of modelling highlights the need for reflective practice in PSM facilitation, allowing researchers to learn from successes and failures and apply those learnings to future projects (13). Voinov et al (13) highlight the importance of keeping detailed documentation of facilitation processes to encourage critical reflection and learning. The use of field notes and a reflective diary in our study significantly helped researchers in this respect by identifying examples of good practice in facilitation but also examples where practice could be improved in the future.

What is the value of PSM in developing a complex systems model for youth suicide prevention in North-West Melbourne?

The value of adopting a participatory approach in developing the North-West Melbourne systems model is demonstrated in the collaborative learning that took place during the PSM. Despite the presence of epistemological differences among stakeholders, engaging in dialogue and debate -within the context of effective facilitation- was invaluable in fostering mutual learning and trust in the PSM. Bringing together different but complementary types of knowledge promoted a sense of shared ownership described by participants as “hope instilling”. Previous research (28) exploring stakeholders’ experiences of engaging in participatory modelling projects has also identified collaboration and co-production as highly valued elements of the participatory process enabling interdisciplinary communication and knowledge exchange.

Central to the process of fostering collaborative learning was the role of lived and living experience. Creating a safe space for young people to draw upon their own experience shaped our understanding of how the system works (and does not work) while at the same time affirmed young people’s capacity to act as creators of knowledge. This has important implications for addressing epistemic injustice (29) in suicide prevention research whereby the voices of those with experiential knowledge, and specifically those of young people, are excluded or silenced due to concerns about inflicting harm and causing distress (17). The present study, however, demonstrates that it is feasible, valuable, and safe to include young people with lived and living experience of self-harm and/or suicidal behaviour in participatory systems modelling projects. Consulting with young people about potential safety issues and ways of facilitating their continued engagement, co-creating safety protocols tailored to their needs and ensuring the involvement of a youth advocate, were key strategies which facilitated the creation of a safe and inclusive space for lived experience involvement. Within the context of repeated calls for addressing epistemic and structural injustice in mental health research (30, 31), we highlight the great potential of participatory systems modelling in legitimising lived experience knowledge through the creation of inclusive and ethical research practices.

What is the impact of the PSM on the model development process?

Stakeholders’ unique knowledge of the regional mental health system informed the structure, logic and assumptions of the systems model thereby increasing its face validity. Contextual knowledge (professional and experiential) of service capacity issues, service access delays, resources and affordability helped identify weaknesses in the system and ways of intervening to improve system behaviour through the identification of locally relevant interventions and policies. Stakeholders also helped the research team identify data gaps and successfully negotiate data access for model calibration and validation.

Stakeholder feedback was considered throughout the project to enhance the PSM process. Young people’s responses to the briefing questionnaires provided useful insights into factors that might affect their participation but also suggestions for ensuring their continued involvement, which the research team implemented (e.g., having a youth advocate). Post-workshop feedback enabled the iterative improvement of the PSM process by reviewing the format and delivery of the workshops (e.g., shorter duration; increasing the number of youth stakeholders) while at the same time reflecting on aspects of the participatory approach that were considered effective (e.g., facilitation) in enhancing stakeholder engagement. Lee et al (19), in their evaluation of a youth mental health PSM process in ACT, also highlight the importance of reflective practice in improving stakeholder experience of and engagement in PSM.

Unpacking the black box of the participatory systems modelling approach

There is no gold standard in how PSM can best be implemented when developing complex systems models (16). However, there are mechanisms, processes and principles which interact to create the optimal circumstances for conducting PSM in a way that is feasible, valuable, and safe. Accurate documentation and reporting of these mechanisms taking place prior to, during and after the modelling process is key in understanding how we can move from nominal to transformative forms of PSM (16).

In their systematic assessment of Group Modelling Building interventions, Rouwette et al (32) adopt Pawson and Tilley’s (33) framework of Context—Mechanism—Outcome configurations (CMO) to capture a comprehensive set of variables that could determine the success of Group Modelling Building interventions. Using this framework and focusing on those variables most relevant and transferable to PSM, we present a theoretical framework of the SEYMOUR PSM approach informed by the findings of the present study (Table 5). This framework explores the relationship between the context, mechanisms, and outcomes of the PSM, offering novel insights into the processes adopted to ensure the safe and inclusive involvement of young people with lived and living experience of self-harm and/or suicidal behaviour.

[INSERT Table 5 HERE]

Context refers to all those pre-existing factors that facilitated the implementation of the PSM including: i) the wider policy context supporting the reform of the mental health system in Victoria, ii) alignment between the project aims and the primary partner agency’s vision for improved suicide prevention for young people in the region, iii) stakeholder characteristics, experience, and roles, and iv) organisation characteristics.

Mechanisms refer to the components, resources or processes in the study which underlie or support the effective delivery of PSM approach (32) and include:

A) Pre-workshop activities (i.e., briefing sessions) to establish young peoples’ motives for taking part and identify potential concerns and ways of ensuring their continued participation.

B) The different processes implemented during the model building workshops, from practical arrangements and logistics to stakeholder knowledge elicitation and sharing, examples of facilitative leadership, creating a safe space for lived experience involvement, and generating and sustaining inclusivity.

C) Post-workshop activities to solicit and enact feedback.

Outcomes, referring to the intended and unintended effects of the PSM, are based on the context and mechanisms interaction and are categorised into individual (e.g., building of trust between research team and expert stakeholder group), group (e.g., shared vision of the problem) and project (e.g., prioritisation of interventions for testing, ensuring model face validity).

This framework will form the basis for future testing and refinement through in-depth qualitative interviews with members of the expert stakeholder group who participated in this study (ongoing work). This framework, which serves as the architecture blueprint of the PSM approach we adopted, has the potential to delineate the key ingredients for the successful delivery of a PSM project which could be transferable and applicable to other system modelling projects in the field of mental health and suicide prevention.

Limitations

The sample size of the participatory model development workshops could be considered relatively low compared to similar studies (19). Whilst the aim was to achieve experiential and demographic variation, our sample allowed for meaningful engagement with key stakeholders, ensuring that conversations were carried out in a safe, respectful and manageable way (34). Indeed, stakeholder feedback highlighted the small group format as suitable for having meaningful conversations.

In line with previous studies in the field of youth mental health and/or suicide prevention (11), our priority was to meaningfully embed local and contextual knowledge into the participant recruitment process. We acknowledge the lack of cultural diversity within our expert stakeholder group and highlight the importance of working closely with local youth organisations and community leaders to diversify recruitment and ensure young people’s needs are sufficiently represented in expert stakeholder groups. Holding workshops in community settings (e.g. youth clubs) could improve inclusivity and accessibility. The debrief questionnaires were not completed by all participants despite several reminders, as a result the feedback might not reflect everyone’s experience. Requesting that stakeholders complete evaluation questionnaires on-site upon completion of each workshop might improve completion rates.

Novel methodological approaches are necessary to improve suicide prevention and intervention efforts and guide effective policy action (4). Such approaches need to span disciplinary boundaries and epistemological silos. Our study shows that participatory systems modelling enables the involvement of multiple, diverse, and even contrasting perspectives in collaboratively developing and evaluating complex systems models to inform evidence-based policy. In doing so, PSM “democratises expertise” (35) by placing experiential knowledge on an equal footing with professional knowledge, thereby positioning those with lived and living experience of suicide at the forefront of co-creating “socially robust knowledge” (35).

CMO Context-Mechanism-Outcome

CMI Client Management Interface

NWMPHN North-West Melbourne Primary Health Network

PSM Participatory Systems Modelling

SDM System dynamics modelling

SEYMOUR System Dynamics Modelling for Suicide Prevention

Ethics approval and consent to participate.

The project has received ethical approval by The University of Melbourne Human Research Ethics Committee (2022-22885-25971-4). All participants provided informed consent to the study.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author upon reasonable request.

Competing interests

JO is both Head of Systems Modelling, Simulation & Data Science at the University of Sydney's Brain and Mind Centre and Managing Director of Computer Simulation & Advanced Research Technologies (CSART).

Funding

This work is supported by the European Union H2020-EU.1.3.2. MSCA-IF-GF - Global Fellowships (Grant agreement ID: 101026065) awarded to Dr Maria Michail. JW is funded by the European Union H2020-EU.1.3.2. MSCA-IF-GF - Global Fellowships (Grant agreement ID: 101026065). JR is funded by a National Health and Medical Research Council Investigator Grant (GNT2008460) and a Dame Kate Campbell Fellowship from the Faculty of Medicine, Dentistry and Health Sciences at The University of Melbourne. KW is funded by a National Health and Medical Research Council Investigator Grant (GNT1177787) and a Dame Kate Campbell Fellowship from the Faculty of Medicine, Dentistry and Health Sciences at The University of Melbourne. AS is supported by philanthropic funding from The Grace Fellowship. JO is funded by the National Health and Medical Research Council Centre of Research Excellence “Reducing suicidal thoughts and behaviours in young people presenting for health care” (1171910).

Authors’ contributions

MM, JR, JW conceived the study idea. MM, JR, JW, JO designed the study. MM, KW, DK, MV, ML supported data collection, stakeholder engagement and participant recruitment. MM analysed the data. JW and JR provided methodological input and advice on data analysis. AS supported the interpretation of the technical aspects of modelling. MM drafted the manuscript, and all authors contributed to the edit of the manuscript. All authors read and approved the final version.

Acknowledgements

The authors would like to thank the expert stakeholder group who participated in this study for their insightful contributions, time, and commitment to this work. The authors would also like to thank Dr Juliane Kloess and Dr Sarah-Jane Fenton for their advice on data analysis.

Australian Institute of Health and Welfare. The National Suicide and Self-Harm Monitoring System. Canberra, ACT: AIHW, 2023
Kinchin I, Doran CM. The cost of youth suicide in Australia. Int J Environ Res Public Health 2018;15(4):672.
Department of Health and Human Services, Victorian Government. National Suicide Prevention Strategy for Australia's health system, 2020–2023 https://www.health.gov.au/sites/default/files/2022-12/national-suicide-prevention-strategy-for-australia-s-health-system-2020-2023.pdf [Accessed 6 September 2024]
Michail M., Witt K. Unleashing the potential of systems modeling and simulation in supporting policy-making and resource allocation for suicide prevention. Crisis 2023; 44(4): 261–266
Homer JB, Hirsch GB. System Dynamics modeling for public health: background and opportunities. Am J Public Health 2006; 96:452–8
Atkinson J., Page A., Wells R., Milat A., Wilson A. A modelling tool for policy analysis to support the design of efficient and effective policy responses for complex public health problems. Implement Sci 2015; 10:26
Atkinson J-A, Page A, Prodan A., McDonnell G., Osgood N. Systems modelling tools to support policy and planning. Lancet 2018; 391: 1158–1159
Occhipinti JA, Skinner A, Iorfino F., Lawson K., Sturgess J., Burgess W., Davenport T., Hudson D., Hickie I. Reducing youth suicide: systems modelling and simulation to guide targeted investments across the determinants. BMC Med 2021; 19(1):61 https://doi.org/10.1186/s12916-021-01935-4 Atkinson J.-A., Skinner A., Hackney S., Mason L., Heffernan M., Currier D., King K., & Pirkis J. Systems modelling and simulation to inform strategic decision making for suicide prevention in rural New South Wales (Australia). Aust N Z J Psychiatry 2020; 54(9), 892–901 https://doi.org/10.1177/0004867420932639
Zimmerman L, Lounsbury DW, Rosen CS., Kimerling R., Trafton JA., Lindley SE. Participatory system dynamics modelling: Increasing stakeholder engagement and precision to improve implementation planning in systems. Adm Policy Ment Health 2016; 43(6):834–849
Freebairn L, Occhipinti J, Huntley S. Participatory methods for systems modelling of youth mental health: An implementation protocol. JIMR Res Protoc 2022;11 (2):e32988.
Lee GY, Hickie IB, Occhipinti JA., Song CYJ., Camacho S., Skinner A., Lawson K., Hockey SJ., Hilber AM., Freebairn L. Participatory systems modelling for youth mental health: An evaluation study applying a comprehensive multi-scale framework. Int J Environ Res Public Health 2022;19(7):4015 https://doi.org/10.3390/ijerph19074015
Voinov A., Jenni K., Gray S., Kolagani N., Glynn P. D., Bommel P., Prell C. et al. Tools and methods in participatory modeling: Selecting the right tool for the job. Environ Model Softw. 2018; 109:232–255.
Stave K. Participatory system dynamics modeling for sustainable environmental management: Observations from four cases. Sustainability. 2010; 2(9):2762–2784. https://doi.org/10.3390/su2092762
Voinov A., Bousquet F. Modelling with stakeholders. Environ Model Softw. 2010; 25:11: 1268–1281, https://doi.org/10.1016/j.envsoft.2010.03.007
van Bruggen A, Nikolic I, Kwakkel J. Modeling with stakeholders for transformative change. Sustainability 2019; 11(3):825. https://doi.org/10.3390/su11030825
Michail M, Robinson J, Witt K., Occhipinti JO., Skinner A., Lamblin M., Veresova M., Kartal D., Waring J. Which programmes and policies across health and community settings will generate the most significant impacts for youth suicide prevention in Australia and the UK? Protocol for a systems modelling and simulation study. BMJ Open 2023;13:e071111. https://doi.org/10.1136/bmjopen-2022-071111
Lee GY, Hickie IB, Occhipinti J-A, Song YJC, Skinner A, Camacho S, Lawson K., Hibler AM., Freebairn L. Presenting a comprehensive multi-scale evaluation framework for participatory modelling programs: A scoping review. PLoS ONE 2022; 17(4): e0266125. https://doi.org/10.1371/journal.pone.0266125
Lee GY, Hickie IB, Song YJC, Huntley S, Ho N, Loblay V, Freebairn L, Skinner A, Crosland P, Moore E, et al. Towards youth mental health system reform: An evaluation of participatory systems modelling in the Australian Capital Territory. Systems 2023; 11:8:386. https://doi.org/10.3390/systems11080386
Department of Health North Western Melbourne Primary Health Network (PHN). Needs assessment report November 2020. Available: https://nwmphn.org.au/wp-content/uploads/2020/11/1.1-NWMPHN-Updated_Needs_Assessment_2019.pdf [Accessed 11 December 2023].
State of Victoria. Royal Commission into Victoria’s Mental Health System, Final Report, Summary and recommendations, February 2021
Braun V., Clarke V. Reflecting on reflexive thematic analysis. Qual. Res. Sport Exerc. Health 2019; 11: 589–597
Cloutier C., & Ravasi D. Using tables to enhance trustworthiness in qualitative research. Strateg Organ. 2021; 19:(1): 113–133. https://doi.org/10.1177/1476127020979329
Aceves-Martins M., Aleman-Diaz A., Giralt M., Solà R. Involving young people in health promotion, research and policy-making: practical recommendations. IJQHC, 2019; 31 (2), 147–153, https://doi.org/10.1093/intqhc/mzy113
Freebairn L., Song Y.J.C., Occhipinti JA., Huntley S., Dudgeon P., Robotham J., Lee G. Y., Hockey S., Gallop G., & Hickie I. B. Applying systems approaches to stakeholder and community engagement and knowledge mobilisation in youth mental health system modelling. Int J Ment Health Syst. 2022; 16, 20 https://doi.org/10.1186/s13033-022-00530-1
Voinov L.A., & Gaddisb E.J. Lessons for successful participatory watershed modeling: A perspective from modeling practitioners. Ecol Model 2008; 216: 197–207.
Crompton A., Waring J., Roe B., O’Connor R. Are we all on the same page? A qualitative study of the facilitation challenges associated with the implementation of deliberative priority-setting. Public Manag Rev. 2008; 20(11): 1623–1642, https://doi.org/10.1080/14719037.2017.1417463
Freebairn, L., Atkinson, JA., Kelly, P.M., McDonnell G., Rychetnik L. Decision makers’ experience of participatory dynamic simulation modelling: methods for public health policy. BMC Med Inform Decis Mak. 2018; 18: 131 https://doi.org/10.1186/s12911-018-0707-6
Fricker, M. Evolving concepts of epistemic injustice In: IJ Kidd, J Medina, and G Pohlhaus, editors. The Routledge handbook of epistemic injustice [internet]. 1st ed. New York: Routledge (2017)
Okoroji C., Mackay T., Robotham D., Beckford D., Pinfold V. Epistemic injustice and mental health research: A pragmatic approach to working with lived experience expertise. Front Psychiatry. 2023; 14:1114725, https://doi.org/10.3389/fpsyt.2023.1114725
MacFarlane, A. LeMaster, J. Disrupting patterns of exclusion in participatory spaces: Involving people from vulnerable populations. Health Expect. 2022; 25: 2031–2033. https://doi.org/10.1111/hex.13578
Rouwette, E.A.J.A., Vennix, J.A.M., Mullekom T., VM. Group model building effectiveness: a review of assessment studies. Syst. Dyn. Rev. 2002; 18: 5–45. https://doi.org/10.1002/sdr.229
Pawson R, Tilley N. Realistic evaluation. Sage: London; 1997
Freebairn, L., Atkinson, J., Kelly, P. et al. Simulation modelling as a tool for knowledge mobilisation in health policy settings: a case study protocol. Health Res Policy Sys 2016; 14, 71 https://doi.org/10.1186/s12961-016-0143-y
Nowotny H. Democratising expertise and socially robust knowledge. Sc Public Policy 2003; 30:3: 151–156, https://doi.org/10.3152/147154303781780461

For the purposes of this study, an adverse event was defined as the development of an untoward effect, undesirable clinical occurrence or medical condition, or the deterioration of a pre-existing medical condition, following or during exposure to a study intervention, whether or not considered causally related to the study intervention.
With support from the expert stakeholder group, the research team did manage to engage during the early days of the project a regional Aboriginal and Torres Strait Islander Youth Engagement Worker who had consented to take part in the study to ensure the needs and voices of young Aboriginal and Torres Strait Islander people in the region were appropriately represented. However, this stakeholder left their position and therefore withdrew their consent. The position was not filled during the course of the study.

Tables 1 to 5 are available in the Supplementary Files section.

Competing interest reported. JO is both Head of Systems Modelling, Simulation & Data Science at the University of Sydney's Brain and Mind Centre and Managing Director of Computer Simulation & Advanced Research Technologies (CSART).

Download PDF

Editor assigned by journal
24 Oct, 2024
Submission checks completed at journal
24 Oct, 2024
First submitted to journal
22 Oct, 2024

You are reading this latest preprint version

An evaluation of the feasibility, value and impact of using participatory modelling to inform the development of a regional system dynamics model for youth suicide prevention.

Status:

Version 1

Abstract

Background

Methods

Results

Conclusions

Introduction

Methods

Pre-workshop activities

Model building workshops

Post workshop evaluation

Data analysis

Results

Ways to facilitate effective stakeholder working

Improving diversity and cultural representation within expert stakeholder group

Facilitative leadership in managing power and social dynamics within stakeholder group

Improving inclusivity and accessibility of technical aspects of the modelling

The role of lived experience in shaping and informing the participatory modelling approach

Sharing and exchanging of knowledge

Changes to the model following stakeholder feedback

Stakeholder insight informing model structure and logic

CF1 Yeah, helplines... all sorts of stuff

Identification of interventions and policies of interest for testing

Suggestions for improvement workshop 2

Discussion

What is the impact of the PSM on the model development process?

Conclusion

Abbreviations

Declarations

References

Footnotes

Tables

Additional Declarations

Supplementary Files

Status:

Version 1