Aims and objectives
The overarching aim of the current study is to evaluate the impact of adopting a new Direct to Psychology Stepped Care Model into the multidisciplinary sleep service. The model of care will incorporate a “Direct to Psychology Insomnia” referral pathway, with a “Stepped Care” treatment approach that utilizes a digital intervention, flexibly delivered across the continuum of care. The RE-AIM Framework (20, 21) will be adopted to evaluate individual and service level outcomes in a pre- post study design comparing Standard Care to Direct to Psychology Stepped Care. The objective is to capture all service events, clinical outcomes and quality of life changes for patients referred to the sleep psychology service in order to perform a full evaluation including a cost effectiveness analysis.
The specific hypotheses include the following:
Compared to Standard Care, the Direct to Psychology Stepped Care Model will be associated with:
- Reduced wait times and improved access to treatment
- Equivalent (non-inferior) clinical outcomes and changes in quality of life
- Service delivery that is less resource intensive and delivered at a lower cost
In addition, we hypothesize that stakeholders (referrer, patient, staff) will be satisfied with the service changes.
Design and Setting
This project will employ a single-site, multi-phase, mixed-methods approach to evaluate the
implementation of the new model of care into the Sleep Psychology service in a tertiary public hospital, embedded within the RE-AIM implementation evaluation framework (19, 20, 22). Reporting will align with the Standards for intervention Reporting Implementation Studies (STaRI) framework(23). Patient, service-level and implementation outcomes will be evaluated by comparing patients attending the service from 1 January 2021 until 30 June 2023 as a pre-measure (old model of care) to patients attending the service from implementation of the model from 1 July 2024 to 31 December 2025 as a post-measure (new model of care). This project has received ethics approval via the LNR pathway with a waiver of informed consent with the XX Human Research Ethics Committee (HREC 2022 XXXXX). The Protocol is listed with the Australian and New Zealand Clinical Trials Registry (ACTRN XXXXX). Figure 1 outlines the study flow.
INSERT FIGURE 1 HERE
Figure 1: Study Flow
Participants
Participants are patients, staff and referrers to the sleep centre. The inclusion and exclusion criteria are broad in order to maximize the generalizability and applicability of our results to the real-world health care context.
Patients: Consecutive patients attending the Sleep Psychology service from 1 January 2021.
Inclusion criteria will include all NEW patient referrals to the Sleep Disorders Service at the XXX Hospital who would have been referred to Sleep Psychology at some point in their patient journey from 1 January 2021 until 31 December 2025. Exclusion criteria will include any patients referred prior to 1 January 2021 or after completion of the study.
Staff: Clinicians and team members, who agree to participate in our surveys.
Referrers: All referrers linked to patients included in the project, who respond to our surveys.
Intervention
Standard Care
An exploratory qualitative analysis of consumer and stakeholder perspectives to benchmark the existing service was undertaken, followed by stakeholder consensus groups that informed the new model of care parameters including inclusion and exclusion criteria for the Direct to Psychology Pathway (24), and inclusion and exclusion criteria for assigning patients to the ‘steps’ of the stepped care model (25). In the new Model of Care, all new referrals to the Sleep Disorders Centre will be triaged by a Sleep Physician (as per current practice) and referrals will then be directed to either Standard Care and waitlisted for a Sleep Physician initial consultation, or to the “Direct to Psychology” pathway. Note that referrals to psychology will also be accepted via the Standard Care pathway for patients requiring a medical first contact and directed to the appropriate Stepped Care treatment option.
Direct to Psychology Stepped Care
All patients referred to sleep psychology will complete an initial assessment according to DSM-V-tr and ICD-11 criteria (26, 27), including outcome measures (see Measures). For patients referred ‘Direct to Psychology’ additional measures will manage any potential risks associated with psychology as first contact within the model due to potential ‘missed’ medical clinical indications. These measures will include screening tools for Obstructive Sleep Apnoea (OSA) (using items from the STOP-BANG (28) and Berlin Apnoea (29) Questionnaires), hypersomnolence disorders (using items from the Cataplexy Questionnaire (30) and questions related to the presence of neuromuscular conditions, respiratory conditions and driving risk.
The inclusion criteria for Direct to Psychology criteria as defined in stakeholder co-design activities (24) include;
- being referred for insomnia and/or sleep psychology, and/or
- suboptimal adherence to obstructive sleep apnoea (OSA) treatment including continuous positive airway pressure therapy (CPAP) or mandibular enhancement splint (MAS) if medical management has been already optimised.
Exclusion criteria include;
- significant new/changed sleep disordered breathing (or identified significant sleep disordered breathing)
- highly comorbid patients (e.g., neuromuscular conditions/weakness, respiratory conditions), diagnosis of hypersomnia/question of central disorder of hypersomnolence
- category 1 (urgent) referrals
- driving risk.
Patients identified as meeting the inclusion and exclusion criteria will proceed with Psychology-first intervention as indicated. Patients who meet the exclusion will be redirected back to the Standard Care model and may be re-triaged by the attending medical officer based on the additional information gleaned from the psychological assessment.
Patients progressing to psychological treatment will be matched to the appropriate treatment level. Depending on patient engagement and participation with the treatment level, as well as clinical outcomes on key assessment measures; discharge, on-referral and/or entry to another intervention level will be provided. The proposed steps of care in the new model are outlined in Figure 2.
INSERT FIGURE 2 HERE
Figure 2. Treatment Steps
Level 1: Digital treatment - "Managing Insomnia"
Managing Insomnia is an evidence-based insomnia intervention, free by clinician prescription, developed by "This Way Up" - a digital therapies service through St Vincent's Hospital, Sydney, Australia(17). This therapy consists of 4 sessions, which the client guides themselves through at their own pace. Each session takes about one hour to complete and is designed to be completed at one session per week. Participants have up to 90 days to complete the program once they start. The sessions are presented in a comic-book like format, and patients are prompted to complete questionnaires and sleep diaries throughout the sessions to facilitate engagement with the material. The referring Psychologist is able to track the progress of the patient, including their commencement of the program and completion of each session, as well as their outcome measures collected at each session (Insomnia Severity Index and K-10 score). A brief check-in phone call 2 weeks after treatment prescription will be undertaken to check progress and support engagement with the program as needed. No therapeutic content will be delivered during the phone call (31). If the patient is engaging well with the intervention, a subsequent phone call will be scheduled for another six weeks to coincide with treatment conclusion and to collect outcome measures. If the patient has not commenced the intervention at the initial 2 week check-in, or if the patient reports barriers to engagement, a second check-in phone call will be scheduled 2 weeks later. If the patient has not engaged with the digital therapy within 4 weeks of being prescribed the intervention, reasons for this will be discussed and options for discharge, on-referral and/or entry to another intervention level will be provided.
Level 2: Group program
The manualised CBTi intervention and sleep manual delivered during group intervention was refined by our Sleep Psychology team from an evidence-based insomnia treatment developed at Flinders University, Adelaide, Australia (32). The intervention is 4 sessions and covers the key components of insomnia treatment - sleep restriction therapy, stimulus control, cognitive therapy and relaxation training. Each session is 90 minutes, and there are a maximum of 6 participants in each group session. The experienced Psychologist and Intern Psychologist monitor participation and clinical outcomes during the group program.
Level 3: 1:1 with Sleep Psychology Intern
Patients assigned to this level will undergo treatment 1:1 with the Psychology Intern under supervision from the experienced Psychologist. Psychology interns/trainees have completed undergraduate psychology training and are post-graduate (Masters, Doctorate or PhD) students on clinical placement. The CBTi intervention described above for Level 2 may be provided along with additional evidence-based strategies as required depending on sleep or mental health comorbidities; such as CBT for depression/anxiety, dream rescripting for nightmares, and Motivational Interviewing for Continuous Positive Airway Pressure (CPAP) adherence (33) for those with comorbid OSA. Each of these sessions are 60 minutes in duration and delivered at a weekly to fortnightly interval. The number of sessions depends on how many sessions the patient requires to achieve adequate symptom relief based on ongoing assessment and monitoring. Participation and clinical outcomes are monitored throughout.
Level 4: 1:1 with Experienced Psychologist
The experienced Psychologist will monitor participation and clinical outcomes during the 1:1 intervention program, and provide additional evidence -based strategies depending on sleep or mental health comorbidities (as described in Level 3 above). Discharge or on-referral will be provided at the conclusion of the treatment.
For Levels 2 to 4 - sessions will be delivered flexibly either face-to-face or via telehealth.
Measures
Primary and secondary outcomes will be collected at two endpoints; pre- and post- service change, comparing the pre-implementation period as a pre-measure (old model of care) to those participating in the (NEW) model of care as the post-measure.
Primary outcomes
Service Access Measures: Access measures include new patient activity for the sleep psychology service, and time from referral to treatment commencement and discharge within the sleep psychology service when comparing patients treated under the old model of care compared to those treated under the new model of care. Number of clinical occasions of service will be collected for the sleep psychology service and the medical service and compared for those attending under the old model of care compared to those attending under the new model of care.
Patient-related Outcomes: Patient-related primary outcomes will test non-inferiority in change scores on key clinical outcome measures from pre- to post- intervention when comparing those patients treated under the old model care compared to those treated under the new model of care. Furthermore, % patients treated adequately at each step of the Stepped Care Model will be measured by the number of patients discharged from the service by the treating clinician with outcome measures indicating reliable clinical change from pre- to post- intervention on the clinical outcome measures. The Clinical outcome measures include the Insomnia Severity Index (ISI) (34), Depression Anxiety and Stress Scale (DASS-21) (35), Kessler Psychological Distress Scale (K10)(36), and the EuroQol (EQ-5D-5L) (37).
Insomnia Severity Index (ISI) is a 7-item self-report questionnaire assessing the nature, severity, and impact of insomnia over the past two weeks on a 5-point scale from 0 = no problem; 4 = very severe problem. The measure is widely used in the literature as well as clinically where a score of 11 is indicative of likely meeting criteria for an insomnia disorder in clinical trials and a change score greater than 8 is the minimally important difference to be considered ‘markedly improved’ or in remission (34, 38).
Depression Anxiety and Stress Scale (DASS-21) is a short version of a 42-item self-report instrument designed to measure three related negative emotional states: depression, anxiety and tension/stress over the past 7 days. Items are scored on a 3-point scale from 0 (did not apply to me at all) to 3 (Applied to me very much, or most of the time). Total scores represent symptoms of distress in each of the domains in the ‘normal’, ‘mild’, ‘moderate’, ‘severe’ and ‘extremely severe’ relative to population norms (35).
Kessler Psychological Distress Scale (K10): The K10 is a 10-item self-report scale to assess symptoms over the past 4 weeks. Items are rated on a 5-point scale from 1 (none of the time) to 5 (all of the time). A score of less than 20 represents low psychological distress ‘likely to be mentally well’. Between 20 and 24 represents moderate distress ‘may have a mild mental disorder’ and between 25 and 29 represents high psychological distress ‘likely to have a moderate mental disorder’(36).
EuroQol (EQ-5D-5L): The EQ-5D-5L measures five dimensions of quality of life: mobility, self-care, usual activities, pain/discomfort and anxiety/depression. Each dimension has 5 levels: no problems, slight problems, moderate problems, severe problems, and extreme problems. The EQ VAS records the patient’s self-rated health on a vertical visual analogue scale where the endpoints are labelled ‘The best health you can image (100)’ and ‘The worst health you can image (0)’. The EuroQol also allows preliminary health economics evaluation by performing quality-adjusted life year calculations as a measure of cost-effectiveness of the intervention compared to standard care (37).
Secondary outcomes
Patient satisfaction: To test for non-inferiority in satisfaction with the provided service measured when comparing those patients treated under the old model care compared to those treated under the new model of care, a feedback informed treatment tool will be utilised.
Session Rating Scale (SRS): The Session Rating Scale (SRS) is four-item visual analogue scale with each scale measured between 0 and 10 designed to assess key dimensions of effective therapeutic relationships and global satisfaction with treatment. The SRS is scored by summing score on each of the four scales with a total possible score of 40 (39).
Patient, Staff and Referrer satisfaction: We are particularly interested in incorporating digital intervention into our model of care. Therefore, for those patients assigned to digital intervention in the new model of care, acceptability and relatability of the online program will be measured via post-intervention questionnaires delivered to patients and staff via a RedCap survey. All multidisciplinary staff of the Sleep Disorders Centre employed during the new model of care implementation phase will be send a questionnaire via Microsoft Forms. Note that these questionnaires are modelled on the service evaluation questionnaires from the “Reboot” online pain management program implemented at Queensland Health’s Sunshine Coast Persistent Pain Management Service (31).
Implementation and Evaluation – RE-AIM
To measure the impact of our service change on both patient and service/organisational level outcomes, the evaluation is outlined below with according to the RE-AIM framework (see Table 1).
Table 1: RE-AIM evaluation framework
INSERT TABLE 1 HERE
Statistical Analysis
The project will employ a mixed methods approach. Qualitative surveys for patients, multidisciplinary staff, and referrers was undertaken pre-service change to ascertain the acceptability of the digital therapy, as well as familiarity, access, confidence, and relatability of this treatment option (25). Post-implementation surveys will also be undertaken on these same metrics. Quantitative data will be collected pre- and post-service change to ascertain treatment effectiveness in terms of clinical outcome measures, as well as service measures of interest such as patient flow, occasions of service and program engagement. Based on similar studies(40), and allowing for some incomplete data, we expect to have data for approximately 110 participants from each pre- and post- time period, resulting in a sample of 220 participants overall who will be suitable for the Direct to Psychology pathway. To assess wait times, participants will be categorised into four groups based on the time-period (pre/post implementation) and whether they met eligibility criteria for the Direct to Psychology pathway. Differences in wait times between the groups will be assessed using a one-way ANOVA and linear regression modelling. In a one-way ANOVA study, a sample of 220 participants, divided by 4 groups, achieves 80% power with a significance level of 0.05 to detect a moderate-large effect size (Cohen’s f) of 0.25 (41). The effects of other variables of interest and potential confounders (patient characteristics, diagnosis,) will be tested in multivariable models. For differences in patient outcome measures pre- and post- intervention when comparing pre- and post- service change, the mean between-group difference and 95% confidence interval (CI) will be calculated. As this aspect of the evaluation is a non-inferiority trial (that is, we expect clinical outcomes to not be reduced due to the service change), the criteria adopted to determine non-inferiority will be if the upper bound 95%CI on each clinical outcome measure was less than the minimal clinical important difference (MCID). Quantitative data measuring pre- and post- service change outcomes on key indices of patient service satisfaction, service characteristics (e.g., occasions of service); as well as referrer and staff service satisfaction, and engagement with digital therapies, will be evaluated via univariate statistical tests where indicated such as independent samples t-test (or nonparametric equivalent), and categorical comparisons will be evaluated using Pearson’s chi-square. The distribution of outcome measures will be assessed, and appropriate transformations will be applied for use in modelling. Differences between group means will be tested using generalised linear regression modelling. Health related quality of life will be measured pre- and post- implementation using the EQ-5D-5L. Population norms for EQ-5D-5L Australia will be used to value the data set (42).