Personalized and collaborative care are crucial elements for sustained engagement with digital measurement-based care tools: A mixed-methods study

doi:10.21203/rs.3.rs-4362142/v1

Download PDF

Research Article

Personalized and collaborative care are crucial elements for sustained engagement with digital measurement-based care tools: A mixed-methods study

https://doi.org/10.21203/rs.3.rs-4362142/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background

Measurement-based care (MBC) involves regular symptom monitoring and feedback, enabling early detection of mental health deterioration and allowing clinicians to adjust interventions accordingly. However, its uptake in youth mental health services remains minimal. This study aims to explore perceived barriers and facilitators for sustained use of MBC and the potential of digital technologies to enhance its engagement in young people.

Methods

A mixed-methods study comprising focus groups and surveys was conducted with young people who had previously accessed mental health services. Template analysis was used to analyze qualitative data, and descriptive analyses were performed with quantitative data.

Results

Eighteen young people (mean age = 21.7 years; SD = 3.4) participated in this study. Surveys showed high agreement with the value of MBC (15/18) and a low rate of discomfort with progress evaluation (11/18). Focus groups revealed collaborative data review with clinicians as a key facilitator which fueled future engagement with data inputs and reviews. However, generalization when interpreting data and incomplete representation of mental health by using irrelevant standard measures discouraged the use. Participants suggested digital features for accessibility and data interpretation to enhance consistent engagement with MBC.

Conclusions

The findings indicate that young people’s motivation and sustained engagement in MBC stem from their comprehension of how collected data fosters shared decision-making and personalize treatment. Therefore, regular, collaborative data review sessions are crucial in cultivating a sense of autonomy and purpose in care for young people. This study highlights the importance of data review in enhancing the uptake of MBC, a factor that is often overlooked. By demonstrating this, the study provides a deeper understanding of motivators in MBC and valuable insights for mental health services to explore strategies that regulate data review. To this end, an exemplary way forward would be to develop technologies that facilitates client-clinician communication and empowers young people to advocate for personalized care.

Youth mental health

health services

measurement-based care

feedback informed care

routine outcome monitoring

The prevalence of mental health disorders in young people is increasing, characterized by heterogenous and complex symptom presentations with varying degrees of comorbidities (1–3). Pertinent to their multidimensional needs, measurement-based care (MBC) is a model of care that facilitates the personalization of mental health care through iterative processes of symptom monitoring, evaluation and adjustment of a treatment plan in response to an individual’s progress (4, 5). MBC consists of three components: a) regular collection of self-reported data; b) review of data; and c) use of data. These components of MBC are designed to facilitate data-informed clinical judgements (6, 7), reduce the limitations of clinical intuition (8), and improve overall treatment outcomes by reducing therapeutic variability between clinicians (9). Research shows that MBC reduces symptom severity, increases treatment engagement and efficiency, and improves therapeutic alliance (10–13). Further, MBC is especially effective for individuals who were “not on track” with expected treatment progress as regular monitoring and feedback allowed clinicians to identify and adapt interventions early in treatment (14, 15).

Recent technological developments have increased the accessibility and effectiveness of MBC. Digital platforms enabled easier access to symptom-tracking measures and promptly provided analysis and interpretation of the collected data. Additionally, the interpretation of data using artificial intelligence has the potential to improve diagnosis and prognosis, leading to enhanced personalization of treatment and better outcomes (16, 17). Technology use is considered particularly suitable for young people, the digital natives (18). However, despite its evidenced clinical value, the uptake of MBC in clinical practice is suboptimal and conducted with poor fidelity (19–22).

The challenges of MBC implementation occur at individual, clinician, and organizational levels. Common barriers for clinicians are time constraints (23), cost, and negative attitudes towards the use of standard measures (4, 24). For organizations, staff training and integrating a digital MBC system into an existing database have been found to be challenging (25). Lastly, clients’ concerns included the privacy of data, the inability of a standard measure to provide a holistic representation of mental health, and risks associated with how the data may be used to limit their access to treatments (26, 27). Interestingly, uncertainty around the use of data was a common barrier for clients who engaged with MBC (22, 27). Previous research has indicated that while more frequent use of feedback led to greater treatment outcomes (9, 28), regular review and utilization of collected data were not consistently performed (29, 30). Reviewing progress data with clients is essential in MBC as it provides opportunities for personalization of treatment and encourages shared decision-making (13). Furthermore, collaborative review has been shown to increase client engagement as it allows individuals to effectively communicate their needs to their clinicians (22, 31–33). Review and utilization of data is especially important for young people who value self-autonomy and collaboration in care (32, 34).

This study aims to evaluate young people’s expectations regarding the use of MBC in youth mental health services (35). Specifically, the study seeks to identify perceived facilitators and barriers of MBC to enhance its uptake and fidelity, which is currently underexplored. This study creates a deeper understanding of young people’s perspectives that is posed to enhance engagement and streamline the integration of MBC as part of their mental health care, particularly if it is accounted for in the realisation and implementation of digital solutions.

Study design and setting

This study took place at the Brain and Mind Centre of The University of Sydney, Australia. The study was conducted in two phases. In the first phase, focus groups were conducted to discuss potential motivators of MBC for young people. The second phase, surveys were distributed to the focus group participants (FGPs) to further explore their perspectives towards MBC and the role of digital technology in their mental health care. Acknowledging that sharing in group settings may not be ideal for all participants, this mixed methods approach enabled safe and comfortable participation for all (36).

Participants and recruitment

The study was advertised via posters and student newsletters at the University of Sydney. The recruitment was done passively. Potential participants could freely contact the coauthor MKC. At that point, they were provided detailed information on the study and a consent form All individuals aged between 18 and 30 years with a help-seeking history at mental health services were invited to participate in the study.

Data collection

Focus groups

Focus groups are a form of participatory research that allows participants to express and build on each other’s perspectives (32). Compared to a one-on-one interview, power imbalance between facilitators and attendees can be minimized in a group discussion, making it particularly appropriate for young people (37).

Four focus groups were conducted between August and October 2022, facilitated by two co-authors (MKC and FI). Two focus groups were conducted using the videoconferencing software Zoom, one was in-person, and one was held in a hybrid setting (both in-person and via Zoom). As group discussions can lead to personal and distressful topics, at least one clinical psychologist (SM and MA) was available to provide mental health support upon request. Focus groups were conducted until data saturation was reached. Informed, written consent was obtained prior to focus group participation. All participants received a financial reimbursement for their time.

The focus group agenda was co-designed with young people with lived experience of mental illness to create a safe and open environment, identify any unforeseen shortfalls, and ensure the content was appropriate for targeted attendees (38). Room agreements were established at the beginning in collaboration with participants to develop a safe environment (39). The study goals were communicated during the focus groups to ensure transparency (40). Group discussion questions were developed based on existing literature and modified for appropriateness to young people (32).

The focus groups were structured as follows. First, an introduction and ice-breaking activity was conducted to further create a safe and comfortable environment for participants (39). Then, a presentation on MBC was provided for participants to familiarise them with the purpose, values, and processes of MBC prior to discussing its potential barriers and facilitators. Finally, the group discussion was conducted, utilising the question schedule (Table 1).

Table 1

Focus group question schedule
Questions
• Do you have experience with tracking in general (e.g. steps, sleep, nutrition)? • What facilitated the regular use of the system? • What are the motivating or demotivating factors for young people to routinely monitor their symptoms? • What are the motivating or demotivating factors for young people to review and discuss the collected data with therapist? • Which technological features do you think will facilitate the use of MBC? • What are the 3 essential factors for use of MBC in mental health care?

Surveys

Surveys were distributed to all FGPs after each focus group, via either pen and paper or the online survey software Qualtrics. The survey collected FGPs’ demographic characteristics, their perceptions of MBC. Further, their expectations of the role of technology were investigated to assess the acceptability of digital technologies in mental health services. As young people are often anxious to express their honest opinions in a group environment, the survey allowed participants to express their opinions privately. Survey questions were formulated from existing research on MBC (18, 32, 34).

Data analysis

All focus groups were video-recorded and transcribed verbatim. The transcribed data were analyzed using the qualitative data analysis software NVivo 14 using the template analysis method (41–43). Template analysis was used as the flexibility of the approach does not confine researchers to a specific methodology, such as grounded theory or interpretive phenomenological analysis. Multiple coders developed the template early in the analysis for reliability and validity.

The data analysis process was performed as follows. Two co-authors (MKC and ME) individually read the transcript of a focus group that was perceived to have the richest data in relation to the research question. Independently, the two co-authors coded this focus group data and subsequently met to discuss the codes and initial themes and develop a template. Codes were identified from the data inductively while broadly focusing on perceived barriers and facilitators of MBC. The template was used to guide subsequent analysis. One co-author (MKC) coded the remaining data. The two co-authors (MKC and ME) met frequently to iteratively refine and review new codes and themes identified from subsequent data analyses.

Surveys

Participant demographics

Eighteen participants attended the focus groups and completed a survey. There were 3 to 6 participants in each focus group, and the mean duration was 93 min (SD = 18.9). Most participants were female (15/18), and their mean age was 21.7 years (SD = 3.4; range 18–29 years). The study participants represented a diverse range of background and previous help-seeking history (Additional file 1).

Young people’s perspectives on measurement-based care

In the survey responses, the majority of the participants felt confident to explain MBC to their peers (16/18), and agreed with the value of routine monitoring (15/18) (Fig. 1). Additionally, most did not find progress review confronting (11/18) and anticipated that feedback would be helpful for their mental health (13/18).

Previous use and perceived role of digital technology in mental health care

Regarding the use of technology as a part of mental care, 12 participants had previously used a digital tool. Of those 12 participants, 6 reported their experience as being positive, 4 had neither a positive nor negative experience, and 2 reported having a negative experience. Technology was most used for symptom monitoring (7/12). Participants most frequently identified the tracking feature as one of their top 3 choices for helpful digital mental health features. This feature was followed by meditation tools and online resources (Additional File 2).

Focus groups

Using template analysis, three themes were developed (Fig. 2). Those were regular symptom monitoring, reviewing data with clinician, and helpful digital features. Each theme includes several subthemes. The first two capture barriers and facilitators of MBC as subthemes. The last theme includes subthemes describing enabling technologies to further facilitate MBC use in therapy, as suggested by the participants.

Regular symptom monitoring

Majority of participants indicated that they had experience with either self-directed or clinician-guided symptom tracking. Regular symptom monitoring was perceived as a double-edged sword. While several participants expressed that it was helpful to discover new insights from the trends, they expressed difficulties in doing it consistently, especially when they perceived there was a lack of progress. The following sections present potential barriers and facilitators of regular data input as part of MBC.

Barrier: The Need for Consistent Monitoring

Participants expressed their awareness of the accumulation of symptom data that would potentially allow them to understand symptom triggers, guide decision-making in treatment and motivate constructive behavioral changes. However, many participants expressed that symptom monitoring was easy to forget, mobile notifications were often ignored, and completing the same standard measures felt repetitive.

“The biggest hindrance is doing something regularly every day at the same time.” (FGP1)

This barrier was more prominent during the periods of poor mental health. The act of reflecting and recording their symptoms was perceived as being burdensome and effortful during low moods. Additionally, they expressed difficulty admitting their lack of progress and, as a result, feeling defeated and hopeless. Some participants even expressed that being discouraged by symptom deterioration and feeling guilty about “not doing clinician’s homework” rather had negative effects on their mental health care. Due to these reasons, most participants recalled that their use of symptom tracking only lasted for a short term.

“On those particular days, it felt like I was making the choice to be sad, or like the choice was kind of made for me by having to report it. And I was like, ‘Okay, I guess I’m sad. Just give up’. Even if there was a little bit of fight left in you, you just give up at that moment.” (FGP2)

Barrier: Incomplete Representation of Mental Health

Participants expressed reluctance to record their symptoms due to concerns about the potential risk of data misinterpretation. They highlighted that using standard measures irrelevant to their specific symptoms of concern was meaningless and demotivating. The perceived usefulness of collected data emerged as a crucial factor for their sustained engagement and motivation to input data. Additionally, participants preferred the use of an idiographic (i.e., individualized) set of standard measures for MBC, over a nomothetic (i.e., fixed) set of measures.

“It’s not really useful unless the questions are kind of like tailored to your or your experiences.” (FGP3)

Furthermore, participants expressed their expectations regarding the accuracy and usefulness of the data. They wanted to track multidimensional symptoms that effectively communicated a holistic representation of themselves to clinicians. Acknowledging that quantitative measures often only conveyed a singular aspect of their life, they suggested incorporating psychological and non-psychological measures. Participants nominated sleep quality, social network, and blood markers (e.g., thyroid or iron level) measures. They believed that considering these multidimensional factors, which moderated their mental health, could lead to more accurate clinical judgement and better outcomes.

“My response would be dependent on other factors. I could be in a depressed mood because I have depression, but it also could be because I have a thyroid disorder that causes depression.” (FGP4)

Facilitator: New insights

Symptom trends were deemed to help young people identify patterns and discover triggers that affect their mental health. The insights from the data seemed to increase self-agency and motivated them to engage with self-management strategies, including emotional regulation, change in thinking patterns and behavioral changes. Such increase in self-awareness and knowledge on management strategies increased autonomy in young people and enhanced their confidence in the prevention and management of future symptoms.

“The more information that you can learn about yourself, the more that you become aware of the different tools that might be available to you to help manage yourself. It also helps to manage your future symptoms, so, you’re better equipped to face those potential risks.” (FGP5)

Most importantly, participants highlighted that these positive experiences from symptom monitoring (i.e., understanding the moderators of their mental health and the dynamic nature of their mood fluctuations) would serve as a motivator to sustain symptom tracking even during periods of negative process. Hence, reviewing data to enable young people to gain these insights seemed critical to increase the uptake and sustainability of symptom monitoring.

“If you have enough data over time and you can see that in the past, my mood has dipped and then it started to come out. […] Seeing the trends, you can see what could perhaps get me out of it this time?’” (FGP6)

Facilitator: Purpose and Goal Setting

Young people emphasized the importance of being informed about the rationale and mechanism of action of MBC. As their ultimate motivation to engage in therapy was an improvement of their mental health, understanding how symptom monitoring can help to achieve this was perceived to intrigue their interest, serving as a motivator.

“Just an understanding of the benefits of doing a measurement-based care, understanding what’s going on. Knowing the value of doing a survey and what it’s going to achieve? I’m going to therapy voluntarily, and I want to get better. That’s going to motivate me towards doing it.” (FGP7)

Collaborative goal setting with clinicians, where young people were active contributors in the process, seemed to facilitate continuous engagement with symptom monitoring. Young people reported feeling disengaged when tasks felt imposed. In contrast, participants found that a shared understanding of the purpose and goals with their clinician, along with the ability to track progress together, significantly motivated them.

“Something that we did, which I think was really good was we set goals at the beginning of every term and then at the end of every term, we’d go through, and we check off. So, this both kept me as accountable, gave me kind of what I want to do. But it’s also that I get to see that progress, and how much I’ve achieved.” (FGP4)

Data review with clinician

Clinicians’ rapport with clients and open-mindedness during data dissemination were determining factors for young people’s involvement in data review. Two subthemes were developed.

Facilitator: Collaborative care

Many participants desired a sense of partnership when reviewing data with their clinicians.

The process of clinicians personally reviewing their data was considered to help young people feel valued and respected as “equal players in therapy”. All participants favoured data review during treatment and preferred to be actively involved in clinical decision-making. They wanted to provide context around their symptom scores, evaluate goals according to their circumstances and explore future treatment strategies.

“I believe if someone reviews our data/provides input, it will make us feel more valued and heard, and then encourage us to open up more. I feel if they check up routinely and provide some statistics and specific resources to use while undergoing care would be really great.” (FGP8)

However, it was observed that many participants were hesitant to initiate these discussions themselves. Therefore, clinicians’ initiation of these discussions and expressions of curiosity and open-mindedness were believed to facilitate the active involvement of young people in data review.

“When your clinician starts to like, ask you questions like, Is that okay? How does that sound? Would you like to try this, or should we do something else? Or have you seen this? And when they sort of bring you options, and include you in that decision making? I think that really empowers someone.” (FGP9)

Barrier: Generalization when interpreting data

Although shared decision-making based on collected data was seen as empowering, participants were simultaneously concerned about the potential for clinicians to rely too heavily on the data and thus make biased or invalidating assumptions. Previous experience of clinicians generalizing their mental health without consideration of their personal context was frequently reported during focus group discussions. Poor rapport built from these experiences seems to hinder young people’s future engagement in data review.

“Being able to explain that beyond what the data is showing. So of not this assumption that the clinician is going to look at that and think that everything’s horrible, but they know you are struggling, but it can be explained by x, y and Z. You need a space to actually explain that.” (FGP6)

Additionally, participants highlighted the importance of positive reinforcement from clinicians when reviewing data. A deficit-focused interpretation of data or invalidation of their perceived struggles with mental health was demotivating.

“ It can be invalidating to some people, if they feel like they are not sick enough to make someone feel like […] their thing isn’t serious enough that what they are feeling.” (FGP10)

Helpful digital features

Three features were suggested to facilitate the use of MBC in therapy. Participants wanted technology to improve data access, customization, and interpretation.

Accessibility and customization capacity

Accessibility was highlighted as a critical feature for regular symptom monitoring. Participants emphasized that a seamless user experience, without being “clunky”, would promote regular use of technologies.

“For it to become a habit, it needs to be easy to use and enjoy.” (FGP1)

Participants’ design preferences (i.e. color, language, units of scale) varied. However, they all wanted the flexibility to choose the standard measures that they would use to track with their clinician. Participants wanted to ensure that these measures were relevant to their mental health concerns.

“A young person needs to be able to pick how much involvement they have with it? So, they’re not just taking like one giant questionnaire that covers a bunch of stuff that isn’t relevant to them” (FGP3)

Automated reminders and encouragements

In addition to clinicians’ positive reinforcement during sessions, participants suggested that personalized encouragements based on their treatment progress (e.g., positive messages when deteriorating or celebratory messages when improving) could motivate regular data input. They preferred a progress-based encouragements over outcome-based encouragements, such as congratulatory messages on the number of consecutive days of data entry rather than reaching a specific symptom score.

“Little achievements or it’s like, ‘oh my god, I logged into the app for six days in a row’ […] Small stuff like that. It’s so dumb but so effective” (FGP3)

An automated reminder of the participant’s personal goals for recovery were deemed as motivating as these were expected to remind them of the value of regular monitoring and strengthen a sense of purpose.

Data interpretation and visualization support

Having a sense of ownership and understanding their data were valued among participants. Hence, they wanted technology to allow them to access the collected data and support data interpretation. These features included visualization of symptom progress, explanation of their questionnaire results, and guidelines on what constitutes an optimal score.

“When therapist deems it appropriate, like showing an actual figure, showing your progress […] Even if it’s just like an encouragement, and even if it’s getting worse, like actually being able to reflect, I think that’s really pretty valuable.” (FGP7)

Shared access to data with clinicians was welcomed, as this would allow them to identify changes in mental health and facilitate difficult conversations during sessions. Participants found it challenging to discuss their struggles and to recall experiences and not knowing ‘what to talk about in therapy’. Hence, young people wanted shared access to data with their clinicians so that their progress data would serve as a tool to communicate their mental health without them raising it explicitly.

Beyond observing their symptom trends, young people wanted online resources about available treatments or self-management strategies which were personalised to their mental health concerns. They found that access to information and discussing these options with clinicians would make them feel empowered during data review.

“And maybe if we have something where it sort of gives suggestions on like how to deal with what kind of feeling, it will keep you more informed. Then you can kind of proof of concept to your therapist, and then get like a secondary opinion rather than ‘do this.’” (FGP11)

This study explored factors that influence young people’s engagement with MBC in mental health services and the role of digital technology to enhance its use. Surveys showed that most young people agreed with the value of MBC and wanted technology to support symptom monitoring. Template analysis of focus groups revealed three themes, regular symptom monitoring, data review and helpful digital features. Facilitators of MBC included learning how longitudinal symptom data helped to gain new insights on their mental health and achieve individual goals. Young people seemed to experience this through collaborative data interpretation and goal setting with clinicians. Barriers were related to young people’s concerns around data misrepresenting their mental health. Similarly, suggested digital features focused on functions that enhanced personalization (i.e. purpose reminders, and customization capacity) and the use of their data (i.e. data visualization). This study highlights collaborative data review as a crucial factor that could enhance young people’s engagement with MBC in mental health services.

Data review in Measurement-Based Care

The three components of MBC are interdependent, as the review and use of data (collectively addressed as data review hereafter) rely on data input. However, consistent symptom tracking is a well-established barrier in MBC. Themes developed in this study presents data review as a factor that could overcome this barrier.

Data review ensures the efficacy of MBC by allowing early detection of mental health deterioration and facilitating data-informed care. However, its potential as a motivator for sustained engagement is underexplored. Collaborative data review in MBC creates opportunities for young people to experience the value of regular, longitudinal symptom monitoring (44). Young people in the study expected that positive experiences of using the collected data to identify triggers and stratgize future treatment plans would serve as facilitators for future engagement. They wanted to engage in data review that helped to build shared understandings on personal goals and their current mental health conditions. Young people sought to enhance accountability and therapeutic alliance through collaborative data reviews.

Furthermore, a sense of autonomy may be an underlying drive that enhanced young people’s engagement in MBC. They wanted to personalised MBC, tailored to their unique needs. Consistent with previous research, young people preferred idiographic measures (26) over nomothetic measures. Their suggestions for digital features further support this. They wanted technology that empowered them in their care, such as tools that provide data interpretation and available treatments options. Importantly, they did not see technology as a replacement for clinician interaction but rather, as a tool to empower them as active participants in data review and treatment decision-making.

Implications for MBC practice

Understanding data review as a key motivator for young people's engagement with MBC can inform service implementation and digital MBC system development. Findings indicate that three elements may be crucial: collaborative goal setting from the start fosters engagement, individualized monitoring plans, and digital MBC systems that simplify data entry.

To enhance young people’s engagement, data review should prioritize personalization and shared decision-making. Clinicians should discuss personal goals with young people early on, and collaboratively decide on monitoring frequency and measures considering their ongoing relevance. Scheduling regular feedback sessions can also help to build accountability and strengthen the therapeutic alliance. Additionally, technology can be utilized to address the time-consuming nature of data review (45). Digital MBC systems can facilitate immediate data visualization and offer heuristic support for clinical decision-making (46–48), streamlining interpretation for both clinicians and young people, especially in complex cases.

Further research should focus on developing user-friendly digital MBC systems that enhance client-clinician interaction, collaborative care, and shared decision-making. This could involve user-centered design and participation from various stakeholders to ensure smooth integration of feedback into clinical workflows (28, 49).

Limitations

Participants of this study were recruited within the context of the university. Consequently, the level of education and knowledge may have influenced the outcomes of this study, as participants may not have represented young people of diverse socioeconomic status. Nevertheless, gender, sexual, and cultural diversity shown in the participant demographics ensures that a wide breadth of perspectives was presented. Secondly, the focus group method may have limited the richness of information compared to the interview format. However, conducting interviews would not have allowed multiple perspectives to develop. Furthermore, since only one participant used MBC before participating in the study, the focus group method enabled participants to brainstorm moderators and build on other peoples’ opinions. For future studies, semi-structured interviews of champions in each focus group may allow for a deeper inspection of individual perspectives.

This study demonstrates how collaborative data review can be a powerful motivator for young people's engagement with MBC in mental health services. The findings reveal that its value may extend beyond its role in ensuring treatment efficacy by highlighting its potential for overcoming a key barrier, consistency required in symptom tracking. A sense of autonomy and personalized care, fostered through collaborative data interpretation and goal setting, could empower young people to become active participants in MBC. Future research should focus on refining these systems to enhance client-clinician interactions, collaborative care, and shared decision-making, ultimately improving mental health outcomes.

FGP

Focus group participant

MBC

Measurement-based care

Ethical approval

This project was approved by the Northern Sydney Local Health District Human Research Ethics Committees (HREC/17/HAWKE/480) and conducted in accordance to the Standards for Reporting Qualitative Research (50).

Consent for publication

Not applicable

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Competing interest

IBH is the Co-Director, Health and Policy at the Brain and Mind Centre (BMC) University of Sydney. The BMC operates an early-intervention youth services at Camperdown under contract to headspace. He is the Chief Scientific Advisor to, and a 3.2% equity shareholder in, InnoWell Pty Ltd which aims to transform mental health services through the use of innovative technologies.

A/Prof Elizabeth Scott is Principal Research Fellow at the Brain and Mind Centre, The University of Sydney. She is Discipline Leader of Adult Mental Health, School of Medicine, University of Notre Dame, and a Consultant Psychiatrist. She was the Medical Director, Young Adult Mental Health Unit, St Vincent’s Hospital Darlinghurst until January 2021. She has received honoraria for educational seminars related to the clinical management of depressive disorders supported by Servier, Janssen and Eli-Lilly pharmaceuticals. She has participated in a national advisory board for the antidepressant compound Pristiq, manufactured by Pfizer. She was the National Coordinator of an antidepressant trial sponsored by Servier.

No other competing interests were reported.

Funding

The funding sources of this study have had no input into the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. This work was supported by the Medical Research Future Fund Applied Artificial Intelligence in Health Care grant [MRFAI000097]. IBH was supported by an NHMRC Research Fellowship (511921). FI was supported by the Bill and Patricia Richie Foundation. MKC was supported by Research Training Program Scholarship funded by the Australian government.

Authors’ contributions

MKC drafted the manuscript, ME, SM, IBH, and FI reviewed the first draft and provided comments. MKC and ME conducted data analyses, ME and FI reviewed all analyses. All authors (MKC, ME, SM, FI, AH, SJH, ES, AP, and IBH) substantively edited and approved the final version of the manuscript.

Acknowledgements

We thank all the young people who participated in this study and all the staff in the Youth Mental Health Team at the University of Sydney’s Brain and Mind Centre, past and present, who contributed to this work.

Auerbach RP, Mortier P, Bruffaerts R, Alonso J, Benjet C, Cuijpers P, et al. Mental disorder comorbidity and suicidal thoughts and behaviors in the World Health Organization World Mental Health Surveys International College Student initiative. Int J Methods Psychiatr Res. 2019;28(2):e1752.
Auerbach RP, Mortier P, Bruffaerts R, Alonso J, Benjet C, Cuijpers P, et al. WHO World Mental Health Surveys International College Student Project: Prevalence and distribution of mental disorders. J Abnorm Psychol. 2018;127(7):623-38.
Filia K, Rickwood D, Menssink J, Gao CX, Hetrick S, Parker A, et al. Clinical and functional characteristics of a subsample of young people presenting for primary mental healthcare at headspace services across Australia. Soc Psychiatry Psychiatr Epidemiol. 2021;56(7):1311-23.
Lewis CC, Boyd M, Puspitasari A, Navarro E, Howard J, Kassab H, et al. Implementing Measurement-Based Care in Behavioral Health: A Review. JAMA Psychiatry. 2019;76(3):324-35.
Parikh A, Fristad MA, Axelson D, Krishna R. Evidence Base for Measurement-Based Care in Child and Adolescent Psychiatry. Child Adolesc Psychiatr Clin N Am. 2020;29(4):587-99.
Lutz W, Schwartz B, Delgadillo J. Measurement-Based and Data-Informed Psychological Therapy. Annu Rev Clin Psychol. 2022;18:71-98.
McLeod BD, Jensen-Doss A, Lyon AR, Douglas S, Beidas RS. To Utility and Beyond! Specifying and Advancing the Utility of Measurement-Based Care for Youth. Journal of Clinical Child & Adolescent Psychology. 2022:1-14.
Hatfield D, McCullough L, Frantz SH, Krieger K. Do we know when our clients get worse? an investigation of therapists' ability to detect negative client change. Clin Psychol Psychother. 2010;17(1):25-32.
Delgadillo J, Deisenhofer A-K, Probst T, Shimokawa K, Lambert MJ, Kleinstäuber M. Progress feedback narrows the gap between more and less effective therapists: A therapist effects meta-analysis of clinical trials. Journal of Consulting and Clinical Psychology. 2022;90:559-67.
de Jong K, Conijn JM, Gallagher RAV, Reshetnikova AS, Heij M, Lutz MC. Using progress feedback to improve outcomes and reduce drop-out, treatment duration, and deterioration: A multilevel meta-analysis. Clinical Psychology Review. 2021;85:102002.
Brattland H, Koksvik JM, Burkeland O, Klöckner CA, Lara-Cabrera ML, Miller SD, et al. Does the working alliance mediate the effect of routine outcome monitoring (ROM) and alliance feedback on psychotherapy outcomes? A secondary analysis from a randomized clinical trial. J Couns Psychol. 2019;66(2):234-46.
Delgadillo J, de Jong K, Lucock M, Lutz W, Rubel J, Gilbody S, et al. Feedback-informed treatment versus usual psychological treatment for depression and anxiety: a multisite, open-label, cluster randomised controlled trial. The Lancet Psychiatry. 2018;5(7):564-72.
De Jong K, Timman R, Hakkaart-Van Roijen L, Vermeulen P, Kooiman K, Passchier J, et al. The effect of outcome monitoring feedback to clinicians and patients in short and long-term psychotherapy: A randomized controlled trial. Psychotherapy Research. 2014;24(6):629-39.
Rognstad K, Wentzel-Larsen T, Neumer S-P, Kjøbli J. A Systematic Review and Meta-Analysis of Measurement Feedback Systems in Treatment for Common Mental Health Disorders. Administration and Policy in Mental Health and Mental Health Services Research. 2023;50(2):269-82.
Shimokawa K, Lambert MJ, Smart DW. Enhancing treatment outcome of patients at risk of treatment failure: meta-analytic and mega-analytic review of a psychotherapy quality assurance system. Journal of consulting and clinical psychology. 2010;78(3):298.
Fletcher S, Spittal MJ, Chondros P, Palmer VJ, Chatterton ML, Densley K, et al. Clinical efficacy of a Decision Support Tool (Link-me) to guide intensity of mental health care in primary practice: a pragmatic stratified randomised controlled trial. Lancet Psychiatry. 2021;8(3):202-14.
Bickman L. Improving Mental Health Services: A 50-Year Journey from Randomized Experiments to Artificial Intelligence and Precision Mental Health. Administration and Policy in Mental Health and Mental Health Services Research. 2020;47(5):795-843.
Woerner M, Sams N, Rivera Nales C, Gorstein T, Johnson M, Mosser BA, et al. Generational Perspectives on Technology's Role in Mental Health Care: A Survey of Adults With Lived Mental Health Experience. Front Digit Health. 2022;4:840169.
Jensen-Doss A, Haimes EMB, Smith AM, Lyon AR, Lewis CC, Stanick CF, et al. Monitoring Treatment Progress and Providing Feedback is Viewed Favorably but Rarely Used in Practice. Administration and Policy in Mental Health and Mental Health Services Research. 2018;45(1):48-61.
Fortney JC, Unützer J, Wrenn G, Pyne JM, Smith GR, Schoenbaum M, et al. A Tipping Point for Measurement-Based Care. Psychiatric Services. 2016;68(2):179-88.
Metz MJ, Veerbeek MA, Twisk JWR, van der Feltz-Cornelis CM, de Beurs E, Beekman ATF. Shared decision-making in mental health care using routine outcome monitoring: results of a cluster randomised-controlled trial. Social Psychiatry and Psychiatric Epidemiology. 2019;54(2):209-19.
Börjesson S, Boström PK. "I want to know what it is used for": Clients' perspectives on completing a routine outcome measure (ROM) while undergoing psychotherapy. Psychother Res. 2020;30(3):337-47.
Hovland RT, Moltu C. Making way for a clinical feedback system in the narrow space between sessions: navigating competing demands in complex healthcare settings. International Journal of Mental Health Systems. 2019;13(1):68.
Hall CL, Taylor J, Moldavsky M, Marriott M, Pass S, Newell K, et al. A qualitative process evaluation of electronic session-by-session outcome measurement in child and adolescent mental health services. BMC Psychiatry. 2014;14(1):113.
Steinfeld B, Franklin A, Mercer B, Fraynt R, Simon G. Progress Monitoring in an Integrated Health Care System: Tracking Behavioral Health Vital Signs. Administration and Policy in Mental Health and Mental Health Services Research. 2016;43(3):369-78.
Duong MT, Lyon AR, Ludwig K, Wasse JK, McCauley E. Student perceptions of the acceptability and utility of standardized and idiographic assessment in school mental health. International Journal of Mental Health Promotion. 2016;18(1):49-63.
Solstad SM, Castonguay LG, Moltu C. Patients’ experiences with routine outcome monitoring and clinical feedback systems: A systematic review and synthesis of qualitative empirical literature. Psychotherapy Research. 2019;29(2):157-70.
Bickman L, Douglas SR, De Andrade ARV, Tomlinson M, Gleacher A, Olin S, et al. Implementing a Measurement Feedback System: A Tale of Two Sites. Administration and Policy in Mental Health and Mental Health Services Research. 2016;43(3):410-25.
Kwan B, Rickwood DJ, Brown PM. Factors affecting the implementation of an outcome measurement feedback system in youth mental health settings. Psychotherapy Research. 2021;31(2):171-83.
Bickman L. A measurement feedback system (MFS) is necessary to improve mental health outcomes. J Am Acad Child Adolesc Psychiatry. 2008;47(10):1114-9.
Solstad SM, Kleiven GS, Moltu C. Complexity and potentials of clinical feedback in mental health: an in-depth study of patient processes. Qual Life Res. 2021;30(11):3117-25.
Moltu C, Veseth M, Stefansen J, Nøtnes JC, Skjølberg Å, Binder PE, et al. This is what I need a clinical feedback system to do for me: A qualitative inquiry into therapists' and patients' perspectives. Psychother Res. 2018;28(2):250-63.
Solstad SM, Kleiven GS, Castonguay LG, Moltu C. Clinical dilemmas of routine outcome monitoring and clinical feedback: A qualitative study of patient experiences. Psychotherapy Research. 2021;31(2):200-10.
Lavik KO, Veseth M, Frøysa H, Stefansen J, Nøtnes JC, Moltu C. This is what I need a clinical feedback system to do for me: A qualitative inquiry into perspectives of adolescents and their therapists. J Clin Psychol. 2020.
Hackett CL, Mulvale G, Miatello A. Co-designing for quality: Creating a user-driven tool to improve quality in youth mental health services. Health Expect. 2018;21(6):1013-23.
Bowen DJ, Kreuter M, Spring B, Cofta-Woerpel L, Linnan L, Weiner D, et al. How we design feasibility studies. American journal of preventive medicine. 2009;36(5):452-7.
Bagnoli A, Clark A. Focus groups with young people: a participatory approach to research planning. Journal of Youth Studies. 2010;13(1):101-19.
Tapsell A, Martin KM, Moxham L, Burns S, Perlman D, Patterson C. Expert by Experience Involvement in Mental Health Research: Developing a Wellbeing Brochure for People with Lived Experiences of Mental Illness. Issues in Mental Health Nursing. 2020;41(3):194-200.
Adler K, Salanterä S, Zumstein-Shaha M. Focus Group Interviews in Child, Youth, and Parent Research: An Integrative Literature Review. International Journal of Qualitative Methods. 2019;18:1609406919887274.
Hawke LD, Relihan J, Miller J, McCann E, Rong J, Darnay K, et al. Engaging youth in research planning, design and execution: Practical recommendations for researchers. Health Expectations. 2018;21(6):944-9.
Brooks J, McCluskey S, Turley E, King N. The Utility of Template Analysis in Qualitative Psychology Research. Qual Res Psychol. 2015;12(2):202-22.
Braun V, Clarke V. Using thematic analysis in psychology. Qualitative research in psychology. 2006;3(2):77-101.
Braun V, Clarke V. One size fits all? What counts as quality practice in (reflexive) thematic analysis? Qualitative Research in Psychology. 2021;18(3):328-52.
Hovland RT, Ytrehus S, Mellor-Clark J, Moltu C. How patients and clinicians experience the utility of a personalized clinical feedback system in routine practice. Journal of Clinical Psychology. 2023;n/a(n/a).
Sichel CE, Connors EH. Measurement feedback system implementation in public youth mental health treatment services: a mixed methods analysis. Implement Sci Commun. 2022;3(1):119.
Lee EE, Torous J, De Choudhury M, Depp CA, Graham SA, Kim H-C, et al. Artificial Intelligence for Mental Health Care: Clinical Applications, Barriers, Facilitators, and Artificial Wisdom. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging. 2021;6(9):856-64.
Iorfino F, Occhipinti J-A, Skinner A, Davenport T, Rowe S, Prodan A, et al. The Impact of Technology-Enabled Care Coordination in a Complex Mental Health System: A Local System Dynamics Model. J Med Internet Res. 2021;23(6):e25331.
Ospina-Pinillos L, Davenport T, Iorfino F, Tickell A, Cross S, Scott EM, et al. Using New and Innovative Technologies to Assess Clinical Stage in Early Intervention Youth Mental Health Services: Evaluation Study. J Med Internet Res. 2018;20(9):e259.
Lewis CC, Marti CN, Scott K, Walker MR, Boyd M, Puspitasari A, et al. Standardized Versus Tailored Implementation of Measurement-Based Care for Depression in Community Mental Health Clinics. Psychiatr Serv. 2022;73(10):1094-101.
O'Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Acad Med. 2014;89(9):1245-51.

Competing interest reported. IBH is the Co-Director, Health and Policy at the Brain and Mind Centre (BMC) University of Sydney. The BMC operates an early-intervention youth services at Camperdown under contract to headspace. He is the Chief Scientific Advisor to, and a 3.2% equity shareholder in, InnoWell Pty Ltd which aims to transform mental health services through the use of innovative technologies.

No other competing interests were reported.

Download PDF

Editorial decision: Revision requested
07 May, 2024
Editor assigned by journal
06 May, 2024
Submission checks completed at journal
06 May, 2024
First submitted to journal
03 May, 2024

You are reading this latest preprint version

Personalized and collaborative care are crucial elements for sustained engagement with digital measurement-based care tools: A mixed-methods study

Status:

Version 1

Abstract

Background

Methods

Results

Conclusions

Figures

INTRODUCTION

METHODS

Study design and setting

Participants and recruitment

Data collection

Focus groups

Surveys

Data analysis

RESULTS

Surveys

Participant demographics

Young people’s perspectives on measurement-based care

Previous use and perceived role of digital technology in mental health care

Focus groups

Regular symptom monitoring

Barrier: The Need for Consistent Monitoring

Barrier: Incomplete Representation of Mental Health

Facilitator: New insights

Facilitator: Purpose and Goal Setting

Data review with clinician

Facilitator: Collaborative care

Barrier: Generalization when interpreting data

Helpful digital features

Accessibility and customization capacity

Automated reminders and encouragements

Data interpretation and visualization support

DISCUSSION

Data review in Measurement-Based Care

Implications for MBC practice

Limitations

CONCLUSIONS

Abbreviations

Declarations

References

Additional Declarations

Supplementary Files

Status:

Version 1