Towards Youth Mental Health System Reform: An Evaluation of Participatory Systems Modelling in the Australian Capital Territory

Preprint

Article

Towards Youth Mental Health System Reform: An Evaluation of Participatory Systems Modelling in the Australian Capital Territory

Altmetrics

Downloads

289

Views

Comments

Submitted:

26 June 2023

Posted:

26 June 2023

You are already at the latest version

A peer-reviewed article of this preprint also exists.

Abstract

Australia’s mental health system is failing young people. Calls for accountability, strategic long-term policy planning, and regional leadership have been identified as solutions to guide mental health reform. Developing system dynamics models using a participatory approach (participatory systems modelling, PSM) is recognised as a useful method that can support decision-making for strategic reform. This paper reports evaluation findings of a youth mental health PSM process conducted in the Australian Capital Territory (ACT). Baseline and follow-up mixed-methods evaluation data were collected in 2022 across diverse stakeholder groups to investigate the feasibility, value, impact, and sustainability of PSM. Though youth mental health system reform was viewed as desirable and a necessity across all stakeholder groups, shared perceptions of disabling powerless was observed regarding their ability to influence current decision-making processes to improve the youth mental health system. This suggests greater accountability is required to support systemic reform in youth mental health. PSM offers promise in improving transparency and accountability of decision-making for youth mental health as exemplified in the ACT. However, more support and time are required to facilitate transformational change. Future research should investigate empowerment strategies to complement the implementation of findings from dynamic models developed through PSM, as well as the effectiveness of regional youth mental health policy decision-making supported by systems modelling.

Keywords:

Subject:

Public Health and Healthcare - Health Policy and Services

1. Introduction

Australia is one of the wealthiest nations, ranking third among the world’s largest economies in 2022,⁽¹⁾ and one of only seven economies recognised as high-income in the Western Pacific region by the World Bank Group for the current 2023 fiscal year.⁽²⁾ Despite no shortage of wealth by global standards, Australia’s mental health system is failing to meet the needs of those with mental ill-health, resulting in at least 55 high-profile public inquiries between 1991 and 2021.⁽³⁾ These resource intensive inquiries have predominantly been initiated by the federal, state, and/or territory governments; however, as there is no obligation that recommendations delivered by the inquiries are implemented, many remain unfulfilled.^{(3, 4)} Australia’s mental health system has consequently been described a “failure in need of treatment”,⁽⁵⁾ faring poorly in comparison to other countries in relation to important systemic markers including suicide and hospital readmissions,^{(6, 7)} leaving many to wonder why the system is so inadequate in such a wealthy nation.

The youth mental health system is particularly in crisis, with concerns increasing due to recent events that continue to undermine the mental health and wellbeing of young people, such as the impacts of COVID-19, extreme weather events causing disruption to education and employment, and economic instability.⁽⁸⁾ A projected 1,093,000 young Australians experienced mental illness within a 12-month period in 2020, ⁽⁸⁾ costing approximately $AUD2 billion in health care expenditure for those under the age of 25.⁽⁹⁾ A substantial amount of mental health care costs for children and adolescents are borne by individuals and their families, with out-of-pocket costs for subsidised services and medicines estimated to be 16%.⁽¹⁰⁾ Out-of-pocket costs are likely to be much higher for non-subsidised services and medicines.⁽¹¹⁾

Youth mental health system reform is therefore critical. Not only are significant investments needed to restore the most productive years of life,⁽¹²⁾ performance and accountability mechanisms are required to ensure funding is better prioritised and importantly, its outcomes assessed.⁽⁶⁾ Though there are ongoing efforts to improve the system such as the substantial increase of the provision of care to improve access to treatments,⁽¹³⁾ more transparency and accountability are required. Accountability in mental health has been described as “outcome-blind”, and failure to manage youth mental health challenges has profound and lifelong consequences for young people and their families, communities, and the economy.^{(14, 15)} Thus, continuing to invest in existing or new programs with minimal understanding of their potential impact within an already fragmented system will not propel necessary changes to deliver substantial reductions in mental ill-health in young people,⁽¹⁵⁾ and new solutions are required to achieve genuine youth mental health system reform.

A call for accountability and regionalisation towards strategic long-term policy planning and strengthened regional leadership have been identified as possible solutions to guide mental health system reform.^{(6, 15, 16)} Decentralised planning processes can provide regional communities information, incentives, and accountabilities to ensure local decision makers deliver improved outcomes.⁽⁶⁾ Achieving accountability and integrated regional planning was first proposed as a priority in 1992 in the First National Mental Health Plan, and remains as the top priority area on the current Fifth National Mental Health Plan.⁽¹⁷⁾ Most recently, the Productivity Commission’s Inquiry report on mental health has noted regional planning to be ‘patchy’, with a lack of accountability and strategy undermining impact on mental health outcomes.^{(6, 18-20)} Youth mental health system reform also requires consideration of broader social and economic factors, such as housing and employment, that impact youth mental health outcomes.^{(6, 21)}

Systems modelling via a participatory approach (referred to as participatory systems modelling or PSM) has been recognised as a useful method to support complex systems reform, particularly in the field of environmental sciences.⁽²²⁾ This approach has potential to increase capacity to support youth mental health reform by offering sophisticated forecasting and decision support tools to prospectively simulate mental health reform options through the consideration of complex and interconnected areas of health, social, and economic systems.^{(21, 23)} PSM can also improve accountability through transparent tracking of such mental health reform options.^{(16, 24)} Importantly, the PSM process can strengthen a multi-sectoral response to youth mental health reform by inviting diverse sectors to be part of the participatory process. The inclusion of diverse stakeholders can also improve accountability on regional and national leaders to make more strategic long-term decisions on how best to invest limited funding to improve multi-sectoral system coordination and efficiency to deliver the greatest benefit, better protecting the mental health and wellbeing of young people.^{(23, 25)}

A team of multidisciplinary researchers at the University of Sydney’s Brain and Mind Centre partnered with the Office for Mental Health and Wellbeing (the Office) in 2022 with the aim to strengthen regional decision-making capacity and accountability for youth mental health in the Australian Capital Territory (ACT). Through a PSM process, a system dynamics model (the model) was developed and customised to the regional context of the ACT. Further information regarding engagement with the Office is described in Box 1.

Box 1. Summary of engaging in a PSM process with the Office.

The Office was established in 2018 by the ACT government as a response to Australia’s current Fifth National Mental Health Plan, which highlights the need for strengthened regional mental health systems planning.⁽²⁶⁾ The primary function of the Office is to act as a catalyst for change to improve the mental health and wellbeing of the whole ACT population.⁽²⁶⁾ This includes strengthening youth mental health systems, which has been recognised as a priority area since its establishment,⁽²⁷⁾ with ACT having the second highest rate of suicide deaths per 100,000 in Australia of children and adolescents aged 5-17 between 2017-2021.⁽²⁸⁾ The process of developing the model for the ACT, including key data insights, is described elsewhere.^{(23, 25)} In summary, the model was developed to provide regional leaders with a sophisticated forecasting and strategic decision-support tool to inform investments on how to achieve the greatest population outcomes for young people in the ACT. This model is delivered via an online platform which can simulate real-time hypothetical ‘what-if’ scenarios of likely health, social, and economic impacts of individual (e.g., first scenario in Figure 1) and/or a combination of programs and services (e.g., second scenario in Figure 1).

Figure 1. The ACT system dynamics model interface aims to allow end-users to ask ‘what if’ questions and determine the most impactful strategic investments.

1.1 Objective

The participatory development of a system dynamics model for the ACT aims to support more strategic long-term youth mental health policy planning and regional leadership - both of which have been identified as potential solutions to guide mental health system reform.^{(15, 16)} However, no literature to date has comprehensively evaluated such an approach. Additionally, just as previous mental health inquiries in Australia have sought to learn from the personal stories of individuals who have experienced poor quality care,⁽³⁾ it is critical that young people with lived experience of mental ill-health can meaningfully contribute (i.e., non-tokenistically) throughout the entirety of the PSM process. This paper reports baseline and first follow-up evaluation results specifically examining the feasibility, value, impact, and sustainability of the PSM process in the ACT. The diverse perspectives of stakeholders, including young people with lived experience of mental ill-health, are explored and reported.

2. Materials and Methods

2.1. Research Context

This evaluation is part of a broader five-year national research program, Right care, first time, where you live. The research program aims to strengthen youth mental health systems by supporting regional leaders to make informed decisions so that young people in their region can access the right level of care, timely delivered early in the course of illness.^{(23, 25, 29)} This program will work with eight geographically diverse regions across Australia to develop and implement system dynamics models designed to reflect the needs of each region through a PSM process. Located in Australia’s capital city Canberra, the Office in the ACT was the first site to participate in the program. Though the Office was the primary partner of the University of Sydney’s Brain and Mind Centre, this research program focused on a holistic participatory process which involved diverse stakeholders from across the ACT region. The term ‘site’ thus interchangeably refers to both the Office and the broader ACT region, unless specified.

The evaluation framework and associated research protocols, which describe in detail the conceptual underpinnings of this study, are reported elsewhere.^(29-31) In summary, a comprehensive multi-scale evaluation framework underpins this evaluation study. This framework aims to longitudinally assess the feasibility, value, impact (change & action), and sustainability of PSM processes in the context of the broader youth mental health research program.^(29-31) A flexible and iterative research approach is enabled through participatory action research principles designed to enable each site to determine their own research processes (e.g., determining their own recruitment procedures).⁽²⁹⁾ Participatory action research also facilitates the iterative improvement of the PSM process by acting on key themes identified by site participants throughout the evaluation, allowing them to take greater ownership as co-researchers to meaningfully participate in decisions aimed at improving research implementation to optimise stakeholder engagement.

2.2. Participants & Sampling

To best capture the local ACT mental health context, diverse stakeholders across the youth mental health sector were invited to attend and contribute to three co-design PSM workshops between March and October 2022. As the Office has a strategic role in mental health services in the ACT, they played an integral role in identifying and inviting key stakeholders to participate in the workshops. Active snowball sampling was also deployed and supported by participatory action research to enable all participants to identify additional stakeholders who should be invited to participate. Over the three workshops, stakeholders worked together to collectively nominate what should be incorporated into the system dynamics model, allowing local contextual priorities to be integrated into the final model. Diverse stakeholders were included such as representatives from the Education, Housing, and Drug and Alcohol Services sectors. For the sake of the analysis, the perspectives of four broad stakeholder groups are explored: (i) community support professionals (e.g., formal carers’ network); (ii) front-line health professionals (e.g., clinicians); (iii) health administrators (e.g., policy makers); and (iv) young people with lived experience of mental ill-health (≥14 years). All stakeholders invited to participate in the PSM workshops were invited by the first author (GYL) to contribute to the evaluation.

2.3. Study Design

A mixed-methods approach was adopted to collect evaluation data throughout the PSM process, aligned with the three co-design workshops. Specifically, data was collected through gamified online surveys and semi-structured interviews prior to the first workshop in March 2022 and immediately after the third workshop in October 2022.

The online surveys had four main components: (i) questions to elicit responses regarding priorities for youth mental health system reform in the ACT, (ii) expectations and experiences participating in the PSM process, (iii) social network analysis, and (iv) patient journey mapping (which will be reported elsewhere). To reduce participant burden and increase participation, online surveys were shortened at follow-up by excluding component (ii) as this data was additionally collected through qualitative interviews. To support respondent engagement (particularly among young people), the online surveys incorporated elements of gamification hosted via the Cogniss platform. This included the incorporation of gamified activities for respondents to complete, as opposed to standard survey templates. An example is the social network analysis activity, whereby respondents engaged in a gamified activity to identify services and/or organisations in the ACT they had previously or were currently working in a professional capacity. This question aimed to understand whether professional respondents (i.e., excluding young people with lived experience) experienced changes in interdisciplinary collaboration as a result of the PSM process.

Interviews were conducted online via video or teleconference by GYL and had three main components: (i) questions to elicit greater understanding about the ACT context (e.g., youth mental health challenges, how regional decisions are made that impact youth mental health care, etc.), (ii) motivations, expectations, and experiences participating in the PSM process, and (iii) social network analysis. A fourth component was included in the follow-up interviews, (iv) facilitators, barriers, and experiences using the model.

Additional data was collected via researcher observations and recordings from the three PSM workshops, meetings with local stakeholders outside the PSM workshops, as well as through reflections and field notes written by GYL.

2.4. Data Analysis

Descriptive statistics were utilised to analyse survey data via IBM SPSS Statistics 28. Survey and interview data were triangulated to develop social network diagrams via Gephi. Interviews were audio-recorded and transcribed via Otter.ai, and subsequently checked for quality and de-identified by GYL and OI. Transcriptions were then coded and analysed by GYL via NVivo. Qualitative analysis was guided by a latent, inductive codebook approach to thematic analysis whereby a semi-structured coding process was undertaken, in conjunction with reflective analytical practice.⁽³²⁾ Specifically, GYL independently analysed each interview and progressively developed a codebook. Throughout this process, GYL met with OI on a biweekly basis to reflectively discuss broader themes, stories, patterns, and meaning across the interviews.⁽³²⁾ Analysis was also progressively reviewed by JO, IBH, VL, and LF, where further reflective practice supported continuous refinement of the analysis through open discussions regarding assumptions, interpretations, and coding of the data. This approach was also applied to triangulate data from other sources such as researcher field notes to form a richer understanding of broader determinants that may affect participation in and outcomes of the PSM process. The cross-validation of data was further complemented by participatory action research as part of a broader reflective cycle whereby strong themes and patterns of data were shared with key stakeholders from the site as they emerged to identify best course of actions, prompting continuous improvement throughout the PSM process. Findings from data analysis are reported utilising broad categories (i.e., feasibility, value, impact, and sustainability of the PSM process) extracted from the multi-scale evaluation framework which underpins the study design.

2.5. Research Ethics

This study has been approved by the Sydney Local Health District Human Research Ethics Council (Protocol No X21-0151 & 2021/ETH00553) and by the Aboriginal Health & Medical Research Council of NSW (1875/21). The evaluation tools have also been approved by the Youth Lived Experience Working Group at the University of Sydney’s Brain and Mind Centre.

3. Results

3.1. Demographics

With regards to surveys, 32 of the 37 respondents who consented to participate at baseline fully completed the survey and were hence included in the analysis. For follow-up surveys, the proportion was 26 of the 31 respondents. A total of 24 and 25 participants consented to contribute to baseline and follow-up interviews, respectively. 11 participants fully completed both baseline and follow-up surveys, and 14 participants contributed to baseline and follow-up interviews. Further demographic information is presented in Table 1.

3.2. Feasibility of PSM to Support Youth Mental Health Systems in the ACT

A draft version of the system dynamics model for youth mental health was delivered to the ACT at the third co-design workshops in October 2022. While 87·5% of survey participants (n=28) had no previous experience utilising such a model to support decision-making, diverse stakeholders contributed to the participatory workshop process in which they interacted with the model, discussed implications of the findings for policy and planning, and gave feedback to the research team to further refine the model.

The opportunity to engage and collaborate with others was identified as a key motivator for baseline survey respondents to participate in the PSM process with 75·0% (n=24) selecting “engage with people from other health services and organisations”, and 62·5% (n=20) selecting “opportunities for further collaborations with other health services and organisations”. This finding was further reinforced in the baseline and follow-up interviews across all stakeholder groups where respondents not only noted opportunities for collaboration with others as a key motivator to participate, but as a necessity. Importantly, interview responses suggested that the PSM (co-design) process enabled the time for participants to develop more trust that a robust model would reflect the local ACT context. Figure 2 illustrates how participants emphasised the importance of trust-building processes in model building. These processes included community consultation and co-design, as well as the hands-on participation of young people with lived experience of mental ill-health.

Though opportunities for collaboration were regarded highly by participants, they also noted challenges working with diverse stakeholders. Specifically, participants suggested further training and standardisation amongst workshop facilitators were required to better manage power relationships between diverse stakeholder groups and ensure that the PSM process was more inclusive, accessible, and transparent for all involved. Select stakeholders from the site including young people with lived experience and researchers from the University of Sydney were nominated as co-facilitators during the workshops to guide group discussions and activities. For example, one front-line health professional noted, “it was sometimes hard to get any other ideas across [as] there were one or two people dominating with their ideas” and suggested in future that the “facilitator could do more containing and asking quieter voices if they wanted to add anything.”

Although workshop facilitation to better manage power relationships was identified as an area for improvement, the manner in which facilitators respected all participants was noted as a key motivator to continue contributing throughout the PSM process. A young person with lived experience of mental ill-health shared how their contributions to the PSM process differed from similar co-design workshops they attended:

“… [the project] went past what I was expecting because I was anticipating reluctance like, ‘It’s the system, and we can’t change it.’ But being able to hear people who are actually running the system saying that they want to make it better made me feel like there could be positive change.”

Though most participants conveyed that they felt respected during the process, another area for improvement was how information was presented to stakeholders. Figure 3 demonstrates how modelling language and technical aspects could be alienating for participants, many of whom lacked prior knowledge and experience of modelling. It was therefore “critical” that time was taken to enable participants to understand the value of the tool. In addition to providing less technical explanations, community support professionals and health administrators suggested that demonstrating what the model could tangibly look like at each of the three workshops would improve engagement throughout the PSM process, as stakeholder engagement varied and new registrants attended each workshop.

3.3. Value of PSM for Youth Mental Health System Strengthening in the ACT

Survey participants identified “address the current gaps of youth mental health care in my community” (90·6%, n=29), “improve the health and wellbeing of young people in my community” (87·5%, n=28), and “assist to make better decisions to improve youth mental health treatments or programs” (84·4%, n=27) as the top benefits that they hoped the system dynamics model would provide for the ACT. These responses were further reinforced during interviews, with a health administrator sharing in their baseline interview their hopes for the system dynamics model tool:

“…to make people think about what the options are. Meaning, funds can be directed in the most optimal way rather than wasting money on things that are not available or unlikely to make a difference [in youth mental health].”

Another health administrator noted in their follow-up interview that the model has the potential to “help write better business cases as sometimes we’re left with such a small amount of time to ask for funding that may last years.” The value of technical analyses offered by the model that may not be readily available to stakeholders was identified, with one health administrator sharing:

“We’re asked to provide a cost benefit analysis [for our business cases] that we’re not really equipped to do … Having this [model] that’s quite accessible and can give us that level of detail is great.”

There was consensus in the baseline and follow-up interviews on the importance of improving the health and wellbeing of young people in the ACT through better policy decisions. As interviews allowed for further probing of responses, systemic challenges impacting youth mental health in the ACT were identified. Challenges identified were similar across participant stakeholder groups, with a common theme being that transparency was lacking around system level decision-making regarding youth mental health in the ACT. In Figure 4, quotes from participants occupying different roles within the system convey a lack of transparency around how decisions are made. This suggests a lack of agency, or stakeholders sensing a “barrier” in understanding who has the power to make decisions to strengthen the youth mental health system in the ACT, which was further emphasised in follow-up interviews as discussed below. Further and notably, there were different views about where responsibility for decision-making was placed across the stakeholders. For example, young people with lived experience of mental ill-health placed responsibility on front-line health professionals or community support professionals; professionals placed responsibility on health administrators; and administrators placed responsibility of ultimate decision-making power at a level above their role (such as the Minister for Mental Health or the federal government).

Despite this confusion, participants across all stakeholder groups shared similar views regarding the value of the model to support improved policy decision-making (Figure 5). The model itself was viewed by participants as a tangible outcome to look forward to, incentivising them to be involved in the PSM process to gain access to the model. Additionally, the model was viewed as a credible and easy-to-use tool, with sentiments shared by a health administrator that they “could put my faith into it [the model]” through the evidence-based approach utilised by the research team of including empirical information in the model. Participants also saw value in the participatory action research approach which supported “reflexive practice in action”, where areas for improvement and solutions could be identified and actioned by the site, giving stakeholders more autonomy in the PSM process.

Limitations of the model were noted by participants, including the lack of data infrastructure, not only in the ACT, but nationally. For instance, as population level data in Australia is currently disaggregated by male and female, some participants expressed concern that the model did not completely capture the whole ACT youth population (e.g., transgender, non-binary, etc.). Additionally, as system dynamics modelling provides insights at a whole population level, concerns that the model “doesn’t reflect the marginalised communities and only helps those organisations that are mainstream” were expressed. One community support professional noted that “the lack of data is phenomenal.” Despite these limitations, another community support professional expressed that the PSM process of working with diverse stakeholders from various disciplines and seniority levels supported an environment where participants were “…able to actually highlight in front of some key government people [which] was very beneficial because it does recognise the fact that there are still challenges around these things [data].” These responses not only reinforced systemic issues such as the need to strengthen data infrastructure, but also further highlighted the need for clearer explanations and communication regarding the model to enable participants to better understand both the parameters and the limitations of the model.

3.4. Impact (Change & Action) of PSM to Support Youth Mental Health Systems in the ACT

A health administrator noted tension “between different sectors” during the first workshop, but by the time of the third, noted that “sectors were working better together.” This interview quote aligned with observations made by the first author, where it was written in GYL’s field note entry for workshop one, “I overhead a workshop participant saying to someone, ‘It’s hard to not feel attacked.’” Comparatively, GYL’s field note entry for the final workshop included:

“…when the model insights were presented, you could visibly see how everyone in the room was very engaged. This was a stark difference from the first workshop where I noticed quite a few people on their phones or answering emails during larger group sessions like these.”

This suggests that the PSM process not only leads to a tangible tool - which was recognised as a facilitator for engagement - but it also supports a broader participant engagement and learning process. This was also noted by a health administrator in their follow-up interview, “… the modelling can definitely shape our thinking and is fantastic for looking at various scenarios.”

These qualitative findings were also somewhat reflected in the quantitative survey data. The PSM workshops provided the opportunity for site stakeholders to prioritise broad strategic intervention categories for inclusion in the final system dynamics model (Figure 1), including but not limited to programs and services that target prevention, early intervention, and community-based youth mental health care. As presented in Table 2, means were calculated utilising survey data on how stakeholders rated the importance of broad strategic categories for inclusion in the model. The means shifted toward the direction of the final model insights - for instance, in favour of programs and services that target early mental health intervention and community-based mental intervention programs, in line with the simulations of the model presented to the site at the third (and final) PSM workshop. However, analysis using linear mixed-effects models detected no statistically-significant change in stakeholders’ ratings for all broad strategic categories between baseline and follow-up. These results may be inconclusive due to insufficient sample size. Statistical significance was observed for respondents’ profession in relation to the prevention and early mental health intervention categories. Community support professionals and health administrators were likely to give a higher importance score in their Likert-scale responses for both strategic categories (i.e., early mental health and community-based intervention programs), compared to front-line health professionals and young people with lived experience of mental ill-health.

Another area where shifts were observed was how young people with lived experience of mental ill-health were included in the PSM process. Though participants across all stakeholder groups agreed at baseline that “young people [should be] at the forefront” throughout the PSM process, a lack of lived experience representation was identified after the first workshop. Participatory action research principles underpinning the study design supported not only reflective practice to jointly identify this area for improvement by the research team and the site, but also led to collaborative actions. As explained by a young person with lived experience during their follow-up interview, “At the first workshop, I was the only young person there.” However, this same participant noted positive changes made by the research team (SH, JO) in collaboration with the site (EM, NJ, SL, JB, EB) to engage more broadly with young people with lived experience in the ACT, ensuring their voices were not only included but genuinely integrated into the PSM process (Box 2).

Box 2. Young person with lived of mental ill-health sharing changes made by the research team and the site to engage and include youth lived experience voices into the PSM process.

“… most of the young people had dropped out because of COVID and things like that [at workshop one] … So before workshop two, there was a young people only workshop, which basically replicated the events of workshop one. … Since it was just me and other young people there [at the youth workshop] it was really great getting to bounce off everyone and have that environment where it’s a safe space without stakeholders or other government people. … A lot of the young people who were at the youth workshop obviously came to workshop two. … And then after workshop two, for those who couldn’t come to workshop two or those who felt like they didn’t have a chance to properly say stuff, we had a debrief meeting. And we all had another proper opportunity to discuss our thoughts which was really, really good.”

Young person with lived or living experience of mental ill-health.

The shift to include more young people with lived experience in the PSM process was viewed as “invaluable” by participants across all stakeholder groups, with suggestions to engage with young people with lived experience as early in the PSM process as possible in future. Young people also expressed not only “feeling heard [which] makes a big difference”, but having confidence that the research team was “actually going to do something with my story.” A young person participant shared that contributing to the PSM process differed from other similar co-design forums:

“I’d share my story but it seemed to fall on deaf ears [in other similar co-design forums], whereas this project has really been like, ‘Okay, we’re actually going to use this information to make a difference and to help improve other people’s lives.’”

A health administrator shared how they had noticed the change in youth lived experience participation throughout the PSM process, attributing the involvement of young people to creating the right balance of representation of diverse stakeholder groups at the workshops. This health administrator shared, during their follow-up interview, the methods adopted throughout PSM which supported improved engagement with young people (Box 3). Improved youth engagement led to young people no longer being motivated by financial incentive alone but valuing other outcomes such as feeling heard and respected without “feeling like the need to receive financial reimbursement for my time - that was just a bonus.”

Box 3. Reflection on strategies to improve youth engagement.

“…I think once we got that balance between the sector and the services with the young people in the room, I think it tipped it back. And I think because we did that specific youth [only] workshop where we really made sure that the young people were set up to succeed when they attended workshop two, they knew what they were walking into. So we had youth workers there [at the PSM workshops] so that if they [young people] felt overwhelmed, they had someone to talk to. We made it a really safe space, which I think allowed them [the young people] to feel brave enough to kind of challenge the sector. And they did particularly [challenge the sector] in workshop two, which I was really proud of. … We ended up with quite a beautiful process between young people having very much a loud role in that [PSM process]. … if I was going into this process again, I would absolutely say that there needs to be a large emphasis on connecting with lived experience as early on as possible.”

Health administrator

In addition to youth engagement, participants shared during follow-up interviews that the PSM process created opportunities for further collaboration between workshop attendees. For instance, a front-line health professional shared their experience of the “gems that have come out of that [PSM] process” regarding developing new collaborations:

“…in terms of connection and just people saying, ‘Can you come talk to my team? Can we send our psychiatric registrar to your service to be co-located?’ … The Office also started a [reference] group bringing services together around the issue of youth mental health. … I don’t know whether that’s a direct outcome of this [PSM process], but that’s how I perceived it which I thought was fantastic.”

While the reference group arose from a separate initiative, qualitative responses highlighted that the PSM process helped to strengthen stakeholder relationships as well as the commitment for diverse sectors to work together. These findings align with survey responses, with 53·8% (n=14) of respondents reporting that they “see more opportunities to work with other organisations and/or people after my involvement in the co-design research program”. Subsequently, 38·5% (n=10) of respondents reported that their “current relationships with organisations and/or people have improved”. Social network analysis, utilising the responses of those who completed both baseline and follow-up surveys, triangulated with qualitative interview data, showed more linkages (or collaborations) were formed between stakeholders from baseline (Figure 6a) and follow-up evaluation data collection points (Figure 6b), as depicted through circle size and line thickness changes demonstrating the strength of collaboration between organisations.

Changes were also made to the online surveys during the PSM process, which incorporated elements of gamification to support respondent engagement, particularly with young people with lived experience of mental ill-health. Participatory action research principles supported a process of iteratively enhancing the follow-up surveys based on feedback received at baseline. Overall, 80·8% (n=21) responded that the gamified activities made the survey more enjoyable at follow-up compared to 56·3% at baseline. Interestingly, 100·0% (n=3 at baseline; n=5 at follow-up) of the respondents who identified as young people with lived experience of mental ill-health found gamified activities made the surveys more enjoyable, whereas front-line professionals and community professionals were more likely to find the surveys confusing or too long (Table 3).

3.5. Sustainability of PSM for Youth Mental Health Systems Strengthening in the ACT

Overall, participants expressed during baseline and follow-up interviews that the PSM process was “worthwhile to commit to” as the model had the “potential to assist with advocacy, planning, and policy making.” The uniqueness of the ACT compared to the other states in Australia was highlighted during follow-up interviews, particularly around how “…the ACT is small enough that this [implementing the model] could work.”

One health administrator admitted that there “…is a little bit of doubt, with any model [about how the model will be used].” However, that same participant also shared that “it’s not to say I don’t have confidence in the model.” Another health administrator expressed that the PSM process supported a “narrative that we must fail forward”, suggesting that despite any skepticism towards the model, that the alternative (i.e., current decision-making practices) is not any better “and we have to do something different [to support youth mental health in the ACT].”

Most participants expressed hopefulness that the model would be sustainably used to support decision-making for youth mental health systems policy and planning decisions. Though participants saw value in the PSM process in that “the co-design makes me more willing to use the model because I understand it more”, participants also expressed that more awareness of system dynamics modelling as a method is required in the youth mental health sector, to increase recognition of the model as a legitimate tool to support decision-making in youth mental health policy and planning in the ACT (Figure 7).

In addition to building more awareness and legitimacy of the model as a youth mental health policy and planning tool, participants expressed that “…if it [the model] got into the hands of the right people, it could be really helpful”. However, as discussed earlier, uncertainty where actual responsibility for decision-making lay, who was going to be accountable for making such decisions considering potential limitations of the model, and how to best incorporate the model in decision-making processes was uncovered during follow-up interviews (Figure 8). A health administrator shared their concerns of “defaulting to old and less reliable methods” if those who are perceived to be in power do not trust or understand the model. A front-line health professional also shared their concerns and questioned how stakeholders in the ACT could be “confident that they [the decision makers] will actually implement any of the recommendations [from the model]”, comparing their observation of recommended evidence-based changes repeatedly not being implemented in the youth mental health system to “…our big inquiries that we have by parliament [in Australia]. … they never implement the findings, so what’s the point?”

Strategies identified by stakeholders to facilitate sustainable implementation of the model in the ACT included: i) ongoing adaptations to the model to reflect the dynamic needs of youth mental health in the ACT, so that the model will not “…just die on the vine like most things do at the end of projects”; ii) widespread training on how to use and interpret the model to prevent “incorrect usage of the model” or the possibility that the model “…might be used in a negative way”, and; iii) implementation of the model within diverse local sectors or organisations in the ACT to mitigate potential challenges, including staff turnover, to “champion” the model and support strategic decision-making for youth mental health planning and policy.

4. Discussion

This paper reports on the feasibility, value, impact, and sustainability of PSM in the ACT, exploring diverse perspectives of stakeholders, including young people with lived experience of mental ill-health. In summary, a system dynamics model was delivered in the ACT via a PSM process. While most stakeholders had no previous experience in PSM nor utilising such a model to support decision-making, diverse stakeholders contributed to the development of and discussions on model insights for youth mental health policy and planning.

A lack of transparency in current decision-making processes regarding youth mental health was uncovered in the ACT. The participatory (co-design) process of developing the model with diverse stakeholders - particularly with young people with lived experience of mental ill-health - was viewed as a facilitator to develop trust that the model could support transparency and improved policy decision-making to best allocate limited funding and achieve optimal mental health outcomes for young people in the ACT. There was however confusion among stakeholders at baseline of where responsibility resided for youth mental health system policy decision-making in the ACT. This confusion was also observed in follow-up interview responses, where there was uncertainty among stakeholders regarding where the model would be best implemented to inform decision-making. This suggested that stakeholders felt a lack of agency or powerlessness in their ability to influence current decision-making processes to improve the youth mental health system in the ACT. This also suggests that greater accountability is required to support systemic reform in youth mental health.

This sense of powerlessness expressed is not unique to the ACT and has been identified as an obstacle that continues to undermine mental health system reform in Australia. Described as the ‘yes minister’ culture, those who are perceived as being in power can also feel disempowered by the bureaucracies that operate within Australia’s mental health system, influencing policy success and failure.⁽³³⁾ This may create the passive acceptance that there are no impactful solutions to achieve youth mental health system reform, which may consequently justify inaction. In PSM research, it has been suggested complex systems, such as youth mental health, fail to undergo reform because solutions being sought are only those that are politically popular at a national level, as opposed to considering solutions that can first enact change on a regional level, before going on to effect changes at national and global system scales.⁽²²⁾

While our results found no evidence that the PSM process had (as yet) led to shifts in how stakeholders would prioritise interventions or strategies to support youth mental health care, there was evidence of improved relationships amongst stakeholders and understanding of what the barriers and opportunities are to regional youth mental health reform. This may therefore indicate that the ACT may be in the early stages of “transformative change.”⁽³⁴⁾ Transformative change requires diverse stakeholders to first develop new kinds of capacity and relationships, which then fosters new ways of practice.⁽³⁵⁾ However, many modelling exercises “fail to reach the goal of empowering stakeholders to take ownership of the decision-making process required”, which can hinder transformative change.⁽³⁵⁾

Thus, the focus in the ACT should now be on maintaining that trust and empowering all regional stakeholders - led by the site - to realise their potential on how they could both contribute to and hold regional leaders accountable for regional youth mental health change in order to support the sustainable implementation of the model. National leaders should also be held accountable to support regional models of governance and decision-making, through improvements to systemic challenges that can limit the system dynamics model, such as current data infrastructure and information collection systems.⁽²¹⁾ Strategies on how to support sustainability were identified by stakeholders during follow-up interviews. It is conceivable that over time, with repeated engagement with the model, youth mental health system strengthening priorities would further align with the modelling insights. Therefore, though youth mental health reform is a difficult endeavor, we argue that the PSM process offers advantages over current decision-making processes.

Further research should investigate empowerment strategies that can complement the implementation of such models developed through a PSM process, holding regional and national leaders more accountable to make more transparent and informed decision-making practice. Future research should also further explore the effectiveness of engaging with the use dynamic systems models and its impacts on regional youth mental health policy decision-making, in the hopes that these regional impacts can catalyse both positive and lasting changes to national and global youth mental health system scales.

Limitations

Due to the nature of participatory action research - where a process of reflective practice prompted continuous improvement throughout the PSM process - changes to the evaluation study protocol were made when participant fatigue was noted. Specifically, as participants engaged in three full-day workshops across a six-month period, participating in additional interviews and surveys was noted as an additional burden. To minimise participant burden, follow-up surveys were shortened to exclude Likert-scale response questions that could be asked during the interview, particularly around participant expectations and experiences participating in the PSM process. The implication of this change was that comparative statistical tests were unable to be performed. Additionally, though 100·0% of youth lived experience respondents reported that gamified activities made the surveys more engaging at baseline and follow-up, future studies should consider a more appropriate balance of questions to include within surveys, as gamified activities may lengthen the time it takes to complete the surveys (a barrier for time constrained stakeholders). The authors also acknowledge that the overall sample size is small, with n=32 and N=24 fully completing the baseline survey and interview respectively, and n=26 and N=25 fully completing the follow-up survey and interview. With that said, a response rate of 75·5% and 56·4% was achieved for baseline and follow-up surveys, respectively. Further, the baseline and follow-up interview results demonstrate responses that reached saturation across diverse perspectives of stakeholder groups who contributed to the PSM process.

5. Conclusions

Across 30 years of Australian mental health plans and policies, one of the most enduring challenges for mental health reform has not been the articulation of desirable goals or change, but its implementation.⁽¹⁶⁾ While this paper has focused on describing a case study in the development of PSM, it highlights more generally the complexity and resources required to build new ways of thinking to support local mental health reform. Systems modelling via a participatory approach is not only feasible but can also add value to a regional youth mental health system, as exemplified by our work in the ACT. The inclusion of diverse stakeholders, including young people with lived experience of mental ill-health, in the PSM process facilitated trust towards the final system dynamics model built for the ACT, and enabled all stakeholders to be actively engaged as co-researchers to cyclically identify areas for improvement and suitable actions to support the participatory process itself. Though youth mental health system reform was viewed as both desirable and a necessity across stakeholder groups, a lack of agency or powerlessness was observed by stakeholders on how they could themselves contribute to improve the youth mental health system in the ACT. Similar observations have been reported in other research in Australia and identified as a barrier to mental health system reform. Ongoing collaboration between researchers and regional stakeholders to facilitate the implementation of the insights derived from the model and, perhaps more importantly, to empower stakeholders to realise their potential on how they could contribute to change, will be critical for successful and sustainable implementation of the learnings of the PSM process to support regional youth mental health system reform in the ACT.

Author Contributions

GYL prepared the original draft of the manuscript, including data analysis, and designed the study with academic oversight from JO, IBH, YJCS, VL, and LF. IBH, JO, and YJCS substantially contributed to the conceptualisation of the study, including acquisition of the research program funding. AS and SHP supported GYL during the quantitative analysis of survey data. OI supported GYL during the initial transcript analysis, namely the proofreading of transcripts to ensure accuracy. All authors contributed equally to manuscript review and have agreed to the published version of the manuscript.

Funding

This research is being conducted under the Brain and Mind Centre’s Right care, first time, where you live Program, enabled by a $AUD12.8 million partnership with the BHP Foundation. The Program will develop infrastructure to support decisions relating to advanced mental health, and guide investments and actions to foster the mental health and social and emotional wellbeing of young people in their communities.

Data Availability Statement

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Acknowledgments

The authors would also like to thank Dr Julien Pollack for his contributions to the social network analysis, and the Youth Lived Experience Working Group at The University of Sydney’s Brain and Mind Centre (Jessica Cottle, Skye Choi, Alexis Hutcheon, Samuel J Hockey, Rachael Laidler, Tara Lindsay-Smith, Claudia Perry, Anith Mukherjee, Joshua New, and Annabel Ramsay) for their contributions in giving insightful feedback on the evaluation tools.

Conflicts of Interest

GYL, YJCS, SH, NH, LF, AS, PC, EM, NJ, SL, JB, EB, CV, SR, PM, OI, and SHP declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. IBH is the Co-Director, Health and Policy at the Brain and Mind Centre (BMC) University of Sydney. The BMC operates an early-intervention youth services at Camperdown under contract to headspace. He is the Chief Scientific Advisor to, and a 3.2% equity shareholder in, InnoWell Pty Ltd. InnoWell was formed by the University of Sydney (45% equity) and PwC (Australia; 45% equity) to deliver the $30 M Australian Government-funded Project Synergy (2017-20; a three-year program for the transformation of mental health services) and to lead transformation of mental health services internationally through the use of innovative technologies. VL is a board member for Matana Foundation, a philanthropic organisation that provides funding to programs for disadvantaged young people in Australia. She does not receive any financial benefit for this role. JO is Co-Director of the Mental Wealth Initiative and Head of Systems Modelling, Simulation and Data Science at the University of Sydney’s Brain and Mind Centre. She holds a non-salaried position as Managing Director of Computer Simulation and Advanced Research Technologies (CSART), a registered charity.

Glossary of Terms

ACT	Australian Capital Territory
Model	System dynamics model
PSM	Participatory systems modelling
Research program	Right care, first time, where you live program
The Office	The Office for Mental Health and Wellbein

References

World Economic Outlook: War Sets Back the Global Recovery; International Monetary Fund (IMF): Washington, DC, USA, 2022.
World Bank Country and Lending Groups: The World Bank. Available online: https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups (accessed on 25 January 2023).
Francis, C.J.; Johnson, A.; Wilson, R.L. The personal cost of repetitive mental health inquiries that fail to result in change. Collegian 2022, 29, 728–737. [Google Scholar] [CrossRef]
Rosenberg, S.; Hickie, I. If We’re to have Another Inquiry into Mental Health, It Should look at why the others have been Ignored Melbourne, VIC: The Conversation. 2018. Available online: https://theconversation.com/if-were-to-have-another-inquiry-into-mental-health-it-should-look-at-why-the-others-have-been-ignored-105728 (accessed on 1 February 2023).
Morris, P. The Australian Mental Health Crisis: A system failure in need of treatment: National Association of Practising Psychiatrists. 2021. Available online: https://napp.org.au/2021/04/the-australian-mental-health-crisis-a-system-failure-in-need-of-treatment/ (accessed on cited 25 January 2023).
Rosenberg, S.; Lawson, K.; Hickie, I. Centralisation versus regionalisation: Designing the Sixth National Mental Health Plan. Australian Journal of Public Administration.n/a(n/a). [CrossRef]
Rosenberg, S.; Hickie, I. No gold medals: Assessing Australia’s international mental health performance. Australas Psychiatry. 2019, 27, 36–40. [Google Scholar] [CrossRef]
Australia’s health 2022: data insights. Canberra, ACT: Australian Institute of Health and Welfare; 2022.
Disease expenditure in Australia 2019–20 Canberra: Australian Institute of Health and Welfare. 2022. Available online: https://www.aihw.gov.au/reports/health-welfare-expenditure/disease-expenditure-in-australia-2019-20/contents/about.
Le, L.K.-D.; Shih, S.; Richards-Jones, S.; Chatterton, M.L.; Engel, L.; Stevenson, C.; et al. The cost of Medicare-funded medical and pharmaceutical services for mental disorders in children and adolescents in Australia. PLOS ONE 2021, 16, e0249902. [Google Scholar] [CrossRef] [PubMed]
Rosenberg, S.; Park, S.H.; Hickie, I. Paying the price—out-of-pocket payments for mental health care in Australia. Australian Health Review 2022, 46, 660–666. [Google Scholar] [CrossRef]
McGorry, P.D.; Goldstone, S.D.; Parker, A.G.; Rickwood, D.J.; Hickie, I.B. Cultures for mental health care of young people: an Australian blueprint for reform. The Lancet Psychiatry 2014, 1, 559–568. [Google Scholar] [CrossRef]
Skinner, A.; Occhipinti, J.-A.; Song, Y.J.C.; Hickie, I.B. Population mental health improves with increasing access to treatment: evidence from a dynamic modelling analysis. BMC Psychiatry 2022, 22, 692. [Google Scholar] [CrossRef]
Rosenberg, S.; Hickie, I. Managing madness: Mental health and complexity in public policy. Evidence Base: A Journal of Evidence Reviews in Key Policy Areas 2013, 1–19. [Google Scholar]
Rosenberg, S.; Hickie, I. Shaping Mental Health Reform - Key Tasks for an Incoming Government. Australas Psychiatry 2022, 30, 746–749. [Google Scholar] [CrossRef]
Occhipinti, J.A.; Skinner, A.; Doraiswamy, P.M.; Fox, C.; Herrman, H.; Saxena, S.; et al. Mental health: build predictive models to steer policy. Nature 2021, 597, 633–636. [Google Scholar] [CrossRef]
The Fifth National Mental Health and Suicide Prevention Plan. Canberra ACT: Commonwealth of Australia, represented by the Department of Health; 2017.
National mental health plan. Canberra: Australian Health Ministers’ Advisory Council; 1992.
Productivity Commission, Mental Health, Inquiry Report. Canberra: Productivity Commission; 2020.
Rosenberg, S. Combatting Commission fatigue: what does real reform in mental health look like? Australian Health Review. 2020, 44, 816–817. [Google Scholar] [CrossRef] [PubMed]
Rosenberg, S.; Salvador-Carulla, L.; Meadows, G.; Hickie, I. Fit for Purpose—Re-Designing Australia’s Mental Health Information System. International Journal of Environmental Research and Public Health 2022, 19, 4808. [Google Scholar] [CrossRef] [PubMed]
Voinov, A.A. How wicked are wicked problems and how do we model them. 10th International Congress on Environmental Modelling and Software; Brussels, Belgium. Brussels, Belgium: International Congress on Environmental Modelling and Software; 2020.
Occhipinti, J.-A.; Skinner, A.; Freebairn, L.; Song, Y.J.C.; Ho, N.; Lawson, K.; et al. Which Social, Economic, and Health Sector Strategies Will Deliver the Greatest Impacts for Youth Mental Health and Suicide Prevention? Protocol for an Advanced, Systems Modelling Approach. Frontiers in Psychiatry 2021, 12. [Google Scholar] [CrossRef]
Rosenberg, S.; Hickie, A.M.I. Making better choices about mental health investment: The case for urgent reform of Australia’s Better Access Program. Australian & New Zealand Journal of Psychiatry 2019, 53, 1052–1058. [Google Scholar]
Freebairn, L.; Occhipinti, J.A.; Song, Y.J.C.; Skinner, A.; Lawson, K.; Lee, G.Y.; et al. Participatory Methods for Systems Modeling of Youth Mental Health: Implementation Protocol. JMIR Res. Protoc. 2022, 11, e32988. [Google Scholar] [CrossRef] [PubMed]
ACT Government Response: Productivity Commission’s inquiry into the social and economic benefits of improving mental health. Canberra, ACT: ACT Government; 2019.
Review of Children and Young People in the ACT. Canberra, ACT: Office for Mental Health and Wellbeing (ACT Government); 2020.
Table 13A.55 Suicide deaths and death rate of people aged 5-17 years [Internet]. Productivity Commission 2022. Available online: : https://www.pc.gov.au/ongoing/report-on-government-services/2023/health/services-for-mental-health/rogs-2023-parte-section13-services-for-mental-health-data-tables.xlsx (accessed on cited 22 May 2023).
Lee, G.Y.; Hickie, I.B.; Occhipinti, J.-A.; Song, Y.J.C.; Camacho, S.; Skinner, A.; et al. Participatory Systems Modelling for Youth Mental Health: An Evaluation Study Applying a Comprehensive Multi-Scale Framework. International Journal of Environmental Research and Public Health. 2022, 19, 4015. [Google Scholar] [CrossRef]
Lee, G.Y.; Hickie, I.B.; Occhipinti, J.-A.; Song, Y.J.C.; Skinner, A.; Camacho, S.; et al. Presenting a comprehensive multi-scale evaluation framework for participatory modelling programs: A scoping review. PLOS ONE. 2022, 17, e0266125. [Google Scholar] [CrossRef]
Freebairn, L.; Song, Y.J.C.; Occhipinti, J.-A.; Huntley, S.; Dudgeon, P.; Robotham, J.; et al. Applying systems approaches to stakeholder and community engagement and knowledge mobilisation in youth mental health system modelling. International Journal of Mental Health Systems 2022, 16, 20. [Google Scholar] [CrossRef]
Braun, V.; Clarke, V. Reflecting on reflexive thematic analysis. Qualitative Research in Sport, Exercise and Health 2019, 11, 589–597. [Google Scholar] [CrossRef]
Grace, F.C.; Meurk, C.S.; Head, B.W.; Hall, W.D.; Harris, M.G.; Whiteford, H.A. An analysis of policy success and failure in formal evaluations of Australia’s national mental health strategy (1992–2012). BMC Health Services Research 2017, 17, 374. [Google Scholar] [CrossRef]
Rass, L.; Treur, J.; Kucharska, W.; Wiewiora, A. Adaptive dynamical systems modelling of transformational organizational change with focus on organizational culture and organizational learning. Cognitive Systems Research 2023, 79, 85–108. [Google Scholar] [CrossRef]
van Bruggen, A.; Nikolic, I.; Kwakkel, J. Modeling with Stakeholders for Transformative Change. Sustainability 2019, 11. [Google Scholar] [CrossRef]

Figure 2. Interview quotes across stakeholder groups expressing key motivations to participate in the PSM process.

Figure 3. Interview quotes emphasising the value of clear and non-technical communication.

Figure 4. Interview quotes across stakeholder groups expressing challenges in understanding how decisions are made in the ACT.

Figure 5. Interview quotes across stakeholder groups expressing the value of the PSM process.

Figure 6. Baseline (a) and follow-up (b) social network diagrams. Circles represent nodes (stakeholder organisations) and the lines represent edges (strength of collaboration between stakeholder organisations). Circle size and line thickness are proportional to the strength of the relationship.

Figure 7. Interview quotes across stakeholder groups expressing the need for the model to be recognised as a legitimate tool to support decision-making for youth mental health policy and planning in the ACT.

Figure 8. Interview quotes across stakeholder groups expressing the need for the model to be recognised as a legitimate tool to support decision-making for youth mental health policy and planning in the ACT.

Table 1. Participant demographic information.

		Baseline (Mar 2022)		Follow-up (October 2022)
		n	%	n		%
Online Survey	Completion of surveys Fully completed Partially completed	32 5	86·5% 13·5%		26 5	83·9% 16·1%
	11 respondents fully completed baseline & follow-up surveys
	Age of respondents 14-17 years old 18-24 years old 25-34 years old 35-44 years old 45-54 years old 55-64 years old 65 years old or older	1 3 6 11 12 4 0	2·7% 8·1% 16·2% 29·7% 32·4% 10·8% 0·0%	1 6 5 7 5 7 0		3·2% 19·4% 16·1% 22·6% 16·1% 22·6% 0·0%
	Profession of respondents¹ Community support professional Front-line health professional Health administrator Young people with lived experience Did not disclose	9 11 4 3 5	28·1% 34·4% 12·5% 9·4% 15·6%	6 8 6 5 1		23·1% 30·8% 23·1% 19·2% 3·8%
	Other information of respondents Culturally and Linguistically Diverse LGBTQIA+ Religious and/or spiritual None of the above	5 4 2 22	15·6% 12·5% 6·3% 68·8%	3 7 0 17		11·5% 26·9% 0·0% 65·4%
Interviews	Completion of interviews	24		25
	14 respondents completed baseline & follow-up interviews
	Profession of respondents Community support professional Front-line health professional Health administrator Young people with lived experience	12 1 9 2	50·0% 4·2% 37·5% 8·3%	9 6 6 4		36·0% 24·0% 24·0% 16·0%

¹ Only fully completed responses included.

Table 2. Baseline and follow-up survey results to understand if participant perceptions shifted throughout the PSM process to align with the insights of the final system dynamics model.

Broad Strategic Intervention Category Prioritised by Site Stakeholders	Baseline Mean¹ (SD), N=32	Follow-up Mean (SD), n=25²	Linear Mixed-Effects Model p-value	Estimates of Fixed Effects for p-values <0.05 (SE)
Prevention programs focusing on health and wellbeing (e.g., education programs)	4·59 (0·71)	4·36 (0·70)	p-value for time = 0·13	N/A
			p-value for profession = 0·03	Community professional = 0·10 (0·33)
				Front-line professional = -0·49 (0·32)
				Health administrator = 0·28 (0·35)
				Young people = -0·16 (0·37)
Early mental health intervention programs (e.g., school counselling services)	4·75 (0·67)	4·80 (0·41)	p-value for time = 0·90	N/A
			p-value for profession = 0·05	Community professional = 0·11 (0·27)
				Front-line professional = -0·45 (0·25)
				Health administrator = 0·24 (0·24)
				Young people = -0·03 (0·30)
Community-based mental health intervention programs (e.g., alcohol and other drugs support programs)	4·59 (0·56)	4·76 (0·44)	p-value for time = 0·16	N/A
	4·59 (0·56)	4·76 (0·44)	p-value for profession = 0·17	N/A
Hospital-based mental health intervention programs (e.g., hospital-to-home services)	4·34 (0·70)	4·20 (0·58)	p-value for time = 0·31	N/A
	4·34 (0·70)	4·20 (0·58)	p-value for profession = 0·76	N/A
Cultural programs (e.g., Aboriginal Community Controlled Health Organisations)	4·56 (0·62)	4·52 (0·59)	p-value for time = 0·27	N/A
	4·56 (0·62)	4·52 (0·59)	p-value for profession = 0·64	N/A
Access programs (e.g., creating opportunities to receive better access to care such as after-hours care or shortened waiting times)	4·81 (0·59)	4·64 (0·67)	p-value for time = 0·16	N/A
	4·81 (0·59)	4·64 (0·67)	p-value for profession = 0·59	N/A
Practical employment or educational skills programs (e.g., financial counselling)	4·50 (0·67)	4·36 (0·49)	p-value for time = 0·57	N/A
	4·50 (0·67)	4·36 (0·49)	p-value for profession = 0·15	N/A

¹ Mean = 1 (Not important at all); Mean = 5 (Very important). ² N=26 for completed follow-up surveys; however, one respondent was excluded from analysis as they skipped this question.

Table 3. Perceptions of gamified online survey.

Baseline (N=32) Community support professional, n=9 Front-line health professional, n=11 Health administrator, n=4 Young people, n=3 Did not disclose profession, n=5	Gamified activities made the survey enjoyable	Overall, n=18 (56·3%) Community support professional, n=5 (27·8%) Front-line health professional, n=4 (22·2%) Health administrator, n=3 (16·7%) Young people with lived experience, n=3 (16·7%) Did not disclose, n=3 (16·7%)
	Gamified activities made the survey confusing	Overall, n=10 (31·3%) Community support professional, n=1 (10·0%) Front-line health professional, n=7 (70·0%) Health administrator, n=1 (10·0%) Young people with lived experience, n=0 (0·0%) Did not disclose, n=1 (10·0%)
	The survey was too long	Overall, n=7 (21·9%) Community support professional, n=2 (28·6%) Front-line health professional, n=4 (57·1%) Health administrator, n=0 (0·0%) Young people with lived experience, n=0 (0·0%) Did not disclose, n=1 (14·3%)
	The survey was no different to any other survey	Overall, n=3 (9·4%) Community support professional, n=2 (66·7%) Front-line health professional, n=0 (0·0%) Health administrator, n=1 (33·3%) Young people with lived experience, n=0 (0·0%) Did not disclose, n=0 (0·0%)
Follow-up (N=26) Community support professional, n=6 Front-line health professional, n=8 Health administrator, n=6 Young people, n=5 Did not disclose profession, n=1	Gamified activities made the survey enjoyable	Overall, n=21 (80·8%) Community support professional, n=4 (19·0%) Front-line health professional, n=6 (28·6%) Health administrator, n=6 (28·6%) Young people with lived experience, n=5 (23·8%) Did not disclose, n=0 (0·0%)
	Gamified activities made the survey confusing	Overall, n=2 (7·7%) Community support professional, n=1 (50·0%) Front-line health professional, n=0 (0·0%) Health administrator, n=0 (0·0%) Young people with lived experience, n=0 (0·0%) Did not disclose, n=1 (50·0%)
	The survey was too long	Overall, n=3 (11·5%) Community support professional, n=1 (33·3%) Front-line health professional, n=2 (66·7%) Health administrator, n=0 (0·0%) Young people with lived experience, n=0 (0·0%) Did not disclose, n=0 (0·0%)
	The survey was no different to any other survey	Overall, n=1 (3·8%) Community support professional, n=0 (0·0%) Front-line health professional, n=1 (100·0%) Health administrator, n=0 (0·0%) Young people with lived experience, n=0 (0·0%) Did not disclose, n=0 (0·0%)

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.

Alerts

MDPI Initiatives

Important Links

Choose an area of interest and we will send you notifications of new preprints at your preferred frequency.

Disclaimer

Towards Youth Mental Health System Reform: An Evaluation of Participatory Systems Modelling in the Australian Capital Territory

Abstract

Keywords:

Subject:

1. Introduction

1.1 Objective

2. Materials and Methods

2.1. Research Context

2.2. Participants & Sampling

2.3. Study Design

2.4. Data Analysis

2.5. Research Ethics

3. Results

3.1. Demographics

3.2. Feasibility of PSM to Support Youth Mental Health Systems in the ACT

3.3. Value of PSM for Youth Mental Health System Strengthening in the ACT

3.4. Impact (Change & Action) of PSM to Support Youth Mental Health Systems in the ACT

3.5. Sustainability of PSM for Youth Mental Health Systems Strengthening in the ACT

4. Discussion

Limitations

5. Conclusions

Author Contributions

Funding

Data Availability Statement

Acknowledgments

Conflicts of Interest

Glossary of Terms

References

MDPI Initiatives

Important Links

Subscribe