Design and overall procedure
The project is a cluster-randomized clinical trial following Good Clinical Practice (GCP) principles. Standard Operating Procedures will be developed for key routines, and digital Client Report Files will be stored in the study technical platform provided by Karolinska Core Facilities. Study coordinators will primarily monitor data quality and routines internally. This protocol was devised according to the SPIRIT reporting guidelines (61). Any substantial amendments to this protocol will be tracked, dated and described in trial reports.
Data will be collected at Närhälsan, the provider of public primary care in Västra Götaland, Sweden, consisting of 104 PCCs in both urban and rural areas. This project studies a naturalistic implementation effort initiated by Närhälsan, with external trainers and supervisors procured by the organization. The study will utilize a cluster-randomized design, randomly assigning PCCs to either implement PCBH immediately (EARLY) during the first 5–9-month period or act as control centers and implement PCBH during the second period (DELAYED). Period 1 starts when all PCCs in the EARLY group have scheduled starting dates during on-site consultations. Period 2 begins when all PCCs in the DELAYED group have done the same, within 5 to 9 months after the start of Period 1. Provided sufficient project resources, period 2 can be lengthened to allow for continued data collection. Patient measures will be collected up to two months before Period 1, and some organizational measures up to one year in advance, establishing a baseline for each PCC. As the project, due to the naturalistic approach, has little direct control over how well a PCC implements PCBH, the analysis plan covers several scenarios where varying degrees of model fidelity are considered (see Data analysis plan).
Participating Primary Care Centers and Randomization
The project includes 8 subregions within Närhälsan, each providing 1-4 PCCs that do not currently adhere to PCBH principles but are willing to implement the model and be part of the randomization process. No other inclusion or exclusion criteria were applied. Basic characteristics of PCCs will be collected through semi-structured interviews at the beginning of the study. This includes information such as the number of listed patients, sociodemographic indexes, number of mental health staff and their training, whether there is and the size of any queues for mental health treatment, as well as details about triage, referrals, and teamwork routines.
In total, 17 PCCs serving a total of 187 000 patients have chosen to participate. For a list of sites, see ClinicalTrials.gov registration. Cluster randomization between the EARLY and DELAYED arms will be conducted using the online tool randomization.com in a 1:1 ratio. The Karolinska Trial Alliance (KTA), an independent unit, performs the randomization separately for each subregion without block division to ensure at least one PCC in the EARLY arm per subregion. If the distribution of PCCs between study arms deviates by more than one, KTA will repeat the randomization until the distribution is balanced or deviates by one. Only the results will be shared with the researchers. Any additional PCCs joining the project after subregion randomization but before the start of implementation period 1 will be randomized by KTA using list randomization with even blocks of random sizes determined by KTA and not shared with researchers.
Participant Recruitment and Power Analysis
All patients aged 18 and above who are suitable for behavioral or mental health interventions, regardless of diagnoses and concomitant care, will be invited to participate in the study. This inclusive approach aligns with the naturalistic setting and the broad scope of the PCBH model. Trained research assistants will identify and contact potential participants, providing digital access to written information and a consent form. Participants will be kept blind to the implementation status of PCBH in their respective PCCs, as all are only informed that the study aims to evaluate and improve mental health treatment in primary care.
We estimate that 70% of patients identified and informed about the study will consent to participate. Power calculations aiming for a power of 90% in detecting a difference of d = 0.4 on a superiority test on the primary patient outcome, adjusted for the lower power in a cluster randomized design with 8 PCCs in one group and 9 in the other, estimating the Intracluster Correlation Coefficient (ICC) to 0.01 (62) and a follow-up response rate of 80%, indicate that a total of 476 patients need to be analyzed, 595 patients recruited, and 850 identified and informed.
If PCBH's superiority over traditional care is not established, a secondary aim is to determine if PCBH is non-inferior. Since there is currently no empirically based definition of the minimal clinically important difference for WHODAS-12 (63) and even less so for the 8-item version we use (see below), we will adopt a conservative non-inferiority margin based on recommendations for a minimally relevant clinical effect for depression, with a Cohen’s d of 0.24 (64). To achieve 80% power with the same ICC as mentioned earlier, this analysis would require the inclusion of 815 patients, with a total of 1455 patients identified and informed. However, this is a secondary objective dependent on resource availability. In case the expected rates of inclusion and follow-up response are lower than anticipated, we will consider extending the implementation periods, if feasible, to increase participant enrollment.
Participating primary care staff
Primary care staff members including physicians, nurses, mental health staff, and management at each PCC will be provided with written information about the study. Those who agree to participate will complete questionnaires assessing work environment and PCBH implementation aspects and may be invited to participate in focus groups.
Measures and assessments
Measures and assessments are categorized into implementation, organization, and patient levels. These will be collected through administrative systems, medical records, national registers, staff and patient self-report forms (primarily via a digital platform, via phone when needed) as well as semi-structured interviews conducted by independent research personnel. Table 1 provides an overview of all patient-level self-report questionnaires and timepoints. To enhance participant retention, automated text message reminders are sent for each assessment. Trained research assistants will also proactively contact patients who have not completed their assessments via phone. For participants who wish to discontinue due to time constraints, the option of completing a shorter assessment that includes the most important questionnaires will be provided. As a final option, they will be given the opportunity to complete only the primary patient outcome assessment over the phone.
Implementation outcomes
Overall categorical PCBH model fidelity
To assess fidelity, implementation measures (see Model adherence, obstacles, and facilitators) and semi-structured interviews with PCC managers will be used. Three fidelity levels are established: FID1 represents no PCBH implementation, FID2 represents partial PCBH implementation, and FID3 represents adequate PCBH implementation. Two of the authors (AFvC, JLL), along with independent PCBH experts and supervisors, will determine the timepoints when each PCC transitions between fidelity levels. The primary outcome is the time taken for a PCC to reach FID3 level. Secondary outcomes include whether a PCC reaches FID3 within the preset implementation period and the time taken to reach FID2 level. These categories will be used as a base for some analyses, including the primary analysis for organizational and patient level data (see Data Analysis Plan).
Implementation success
The s-NoMAD questionnaire, which evaluates the integration of complex interventions into daily practice (65), will be administered to nurses, physicians, mental health workers, and leaders to assess implementation success.
Acceptability, Appropriateness and Feasibility
The Acceptability of Intervention Measure (AIM), Intervention Appropriateness Measure (IAM), and Feasibility of Intervention Measure (FIM) (66) are four-item measures of implementation outcomes that are considered particularly relevant to implementation success (67). Profession-specific versions of these will be administered to nurses, physicians, mental health workers and leaders, focusing on key elements of the PCBH model that are particularly relevant to their daily work.
Model adherence, obstacles, and facilitators
- Average Appointment Length (AAL) assesses the average duration of bookable appointments in mental health workers' schedules, measured in minutes.
- Supply of Appointments (SA) measures the number of bookable appointments available per full-time mental health worker on a weekly basis and reflects the capacity and availability of PCBH services.
- Future Capacity (FC) indicates the number of available timeslots for appointments in the coming 4 weeks divided by the total number of timeslots in mental health staff schedules. It provides insights into the PCC’s ability to accommodate new patients.
- Percentage Longer Treatments (PLT) indicates the proportion of patients who have exceeded the standard number of visits (more than 4), signaling a departure from usual PCBH routine.
- The Primary Care Behavioral Health Provider Adherence Questionnaire (PPAQ) (68) assesses the adherence of behavioral health providers to the PCBH model. For this study, it was adapted for the Swedish setting, incorporating forward and backward translation, the addition of new items, and the removal of irrelevant items. To address the teamwork aspect of PCBH, similar questionnaires were developed for physicians and nurses in collaboration with Swedish PCBH experts, utilizing item-level Content Validity Index ratings (69). The resulting scales are named Integrated Behaviors in Primary Care for General Practitioners / Mental Health / Registered Nurses (IBPC-GP, -MH, -RN).
- Levels of Integration Measure (LIM) (70) evaluates healthcare leaders' perception of the PCC's integration level regarding behavioral health staff. Similar to the PPAQ, adaptations have been made to reflect its use in the Swedish healthcare system.
In addition to these measures, PCBH supervisors will provide ratings and structured qualitative descriptions of each PCC's progress in the implementation process after every supervision meeting. This feedback will capture the supervisors' perspectives on obstacles and facilitators encountered during the local implementation processes.
Organizational outcomes
Measures relevant to patient reach
The penetration rate indicates the number of unique patients seen per full-time equivalent mental health professional and will be measured in relation to all listed patients (PR1) as well as patients with a mental health diagnosis (PR2).
Work environment and teamwork among staff
To evaluate any changes in staff work environment, 35 items from the Copenhagen Psychosocial Questionnaire (COPSOQ-III) (71), a measure of psychosocial work environment, will be administered to participating staff. The subscales of Consensus and Collaboration of the IBPC-GP, -RN and -MH scales will be used to measure teamwork at the PCC.
Measures relevant of access of care and wait times
- Average waiting time (AWT, primary organizational outcome) captures the waiting time from the identification of a behavioral health concern to being seen by any clinician. It includes the time between the patient's initial contact (phone call or message) and the visit with any clinician. If the concern is identified during another visit, the wait time is calculated from the visit where the need was identified to the visit with the referred-to profession.
- Average waiting time for a psychosocial intervention (AWT-P) focuses specifically on the waiting time from the identification of a behavioral health concern to receiving any mental health intervention.
- Number of individuals on waiting lists to receive psychosocial interventions (WL).
- Third Next Available Appointment (TNAA) assesses the waiting time until the third next available appointment for a hypothetical patient who contacts the PCC on a given day. It is measured consistently at the same time every week (e.g., Mondays at 9 AM). TNAA provides insights into the access and availability of care, independent of patients' own ability to show up at a given time.
- Average number of visits and phone/video contacts during patients’ period of care.
Cost, health economics and utilization of medication and sick leave
Utilization of medical care and productivity losses will be measured using the Trimbos and iMTA questionnaire on Costs associated with Psychiatric Illness (TiC-P). This includes assessing the number of days on sick leave, productivity loss while at work, productivity loss at home, and medications prescribed and taken. The 5-item EQ5D will be administered for health economic evaluations. Cancelled Visits and No-Shows (CVNS) will be tracked, capturing the number of visits where patients either cancelled late or did not show up. Specialist Referrals (SR) will be recorded as the percentage of patients referred to specialized care and Denied Referrals (DR) will indicate how many of these referrals were not accepted. Patient self-reports will be complemented and combined with national register-level data on sick leave, medications, and health care utilization.
Patient outcomes
The main time-points for all non-register patient measurements are as follows:
- Before or shortly after first contact with the PCC. This is considered the pre-measure timepoint.
- 4-week follow-up (FU4).
- 8-week follow-up (FU8).
- 12-week follow-up (FU12). This timepoint is considered post-measure when applicable.
- One year follow-up (FU52).
Sociodemographic and Clinical data
Social and economic status, education, profession, and employment/activity status will be collected through self-report. The main reason for the visit, the duration of the problem, other health concerns, concomitant care, care consumption, ICD-10 diagnoses and medications will be captured through self-reports and medical records.
Everyday Functioning, Symptoms, and Quality of Life
The primary patient outcome is everyday functioning, assessed through four domains (8 out of 12 items) of the WHO Disability Assessment Schedule 2.0 (WHODAS-12) (72): Life activities, Cognition, Getting along, and Participation. The domains of Mobility and Self-care, which form independent factors and are not expected to change significantly, will be excluded from primary analyses but considered in secondary analyses. A range of very short versions of well-established patient-rated scales for common mental health conditions will be used to measure symptoms. For an overview, see table 1. Each scale will be used to evaluate changes to the specific symptoms measured, as well as for dividing patients into different analysis groups based on problem areas. Additionally, a total index for symptom load will be calculated from all 21 items, with appropriate weighting to account for the varying number of questions across domains. Quality of Life will be measured by the 12-item Brunnsviken Brief Quality of Life (BBQ) (73), which examines the importance and fulfilment of six areas (e.g. spare time quality, creative work, and friendship). Additionally, the 4-item Outcome Rating Scale (ORS) (74) will be used. Complementing these measures is the DAily Routines for Well-being INventory (DARWIN), developed by the research group, which evaluates the frequency of 11 behaviors relevant to mental health, including exercise, regular eating, socializing, and sleep hygiene.
Table 1 Overview of patient self-report questionnaires measuring symptoms, including timepoints.
Questionnaires measuring symptoms
|
Initial visit
|
FU4
|
FU8
|
FU12
|
FU52
|
Patient Health Questionnaire 9-item (PHQ-9)1
|
X
|
|
|
X
|
X
|
Generalized Anxiety Disorder 7-item (GAD-7)1
|
X
|
|
|
X
|
X
|
Patient Health Questionnaire 4-item (PHQ-4)1
|
|
X
|
X
|
|
|
Panic Disorder Severity Scale 2-item (PDSS-SR-2)2
|
X
|
X
|
X
|
X
|
X
|
Social Phobia Inventory – Abbreviated Version (Mini-SPIN-3)1
|
X
|
X
|
X
|
X
|
X
|
Obsessive Compulsive Disorder 3-item (OCD-3)3
|
X
|
X
|
X
|
X
|
X
|
Perceived Stress Scale 2-item (PSS-2)1
|
X
|
X
|
X
|
X
|
X
|
One-item question on perceived stress3
|
X
|
X
|
X
|
X
|
X
|
Karolinska Exhaustion Disorder Scale 3-item (KEDS-3)3
|
X
|
X
|
X
|
X
|
X
|
Insomnia Severity Index 2-item (ISI-2)1
|
X
|
X
|
X
|
X
|
X
|
Short Health Anxiety Inventory 3-item (SHAI-3)2
|
X
|
X
|
X
|
X
|
X
|
The Alcohol Use Disorders Identification Test-Concise (AUDIT-C-3)1
|
X
|
X
|
X
|
X
|
X
|
Pain (One-item rating from 0 to 10)3
|
X
|
X
|
X
|
X
|
X
|
1 Published and empirically tested. 2 Created by factor analytic item-reduction and sensitivity to change analyses on large datasets from previous trials conducted by the research group, not yet published.
3 Created by an expert group, not yet published or empirically tested.
|
Change, Adverse Events and Satisfaction with Care
Patient experiences and perceptions will be assessed using multiple measures. Satisfaction will be evaluated using 4 items from the Client Satisfaction Questionnaire (CSQ) (75), the Session Rating Scale (SRS) (76), and 9 additional items specifically designed for PCBH settings to gauge patients' perceptions of and attitudes toward their care providers. To measure subjective change, the one-item Patient Global Impression - Improvement (PGI-I) (77) scale will be employed. To assess treatment side effects, an Adverse Events questionnaire will be employed, offering both a concise (3 items) and a more extensive version (9 items) at different timepoints, in which patients will have the opportunity to report and describe any unwanted treatment effects or events they may have experienced. Additionally, at the FU12 timepoint, patients will be offered the opportunity to participate in semi-structured interviews to provide detailed insights on their satisfaction, adverse events, treatment content, goals, and perceived changes in symptoms and quality of life. Any reported adverse events that require immediate action will be reported to the responsible PCC.
Treatment Content
Information on which interventions are used within each primary care model will be obtained through medical records. In addition, patients will be asked in interview form what interventions they remember and which strategies they use after treatment.
Primary care models
Care as usual (CAU)
CAU refers to the currently most common way of organizing behavioral and psychological care in Sweden. In accordance with this, all CAU centers, in this study the DELAYED arm, will have on-site clinical psychologists or social workers delivering a variety of talking treatments. Crucially, these functions will not be integrated into the PCC’s workflow but will function as co-located mental health services, in line with the traditional model for delivering psychosocial interventions.(19) Typical routines at CAU centers will involve written referrals from physicians, no direct triage to mental health services and caregivers managing their own schedules and bookings. Interdisciplinary collaboration, if present, is typically done only through scheduled team conferences. Productivity as well as the reach of interventions in relation to mental health is ordinarily low, resulting in extensive wait times and queued referrals. The content of psychological treatment within the CAU model will vary depending on the staff present at each PCC, from supportive counseling to manualized CBT and long-term psychodynamic psychotherapies. The actual content of an individual patient’s care will be assessed retroactively through medical records. Semi-structured interviews with PCC managers will be used to assess that no PCCs in the CAU condition have workflows that are reminiscent of PCBH.
Primary Care Behavioral Health (PCBH)
An obstacle to high quality PCBH research is that there is a variation in the understanding of the aim and key strategies of the model (59). In this project, a version of the PCBH model adjusted for the Swedish health care system (15) will be used. The focus of the implementation effort is to increase access and decrease wait times for mental health interventions as well as increase collaboration between psychologists/social workers and medical staff. The planned changes include introducing direct triage, same-day appointments, clinical pathways, and a stepped care model for interventions as well as shortening appointment lengths and increasing productivity. Importantly, the typical PCBH routine of ‘warm hand-offs’ (patients being offered a visit with a mental health professional directly after a visit with a primary care physician) (22) will not be emphasized, as Swedish health care increasingly leans more on direct triage, where the physician visit is postponed or replaced entirely (78). Another aim is to increase the efficiency of interprofessional collaboration by defining and clarifying the role of the mental health staff, educating other professions in when and how to utilize their services as well as introducing a common language and mindset for discussing mental health issues with both patients and colleagues. Routine coordination, efforts to reach a shared understanding among staff and quick ‘curbside consultations’ will be emphasized over scheduled team conferences. Mental health staff will be taught to assess patients’ problems through a contextual rather than a medical model of mental health and administer brief interventions adapted to the patients’ problem and life situation, as described in the introduction. All mental health staff will receive three days of training followed by six 4-hour supervision sessions over the course of six months. Training and supervision will focus on both organizational and clinical aspects of the model. Additional support for individual PCCs will be considered in the case of low model fidelity. A local implementation group will be established at each PCC, consisting of all mental health staff, managers as well as representatives for physicians, nurses, and other relevant occupations. All other staff will receive 2 hours of training. All training and supervision will be conducted by an external part. One of the authors (JLL) will together with research assistants oversee adherence by monitoring implementation outcomes (see Model adherence, obstacles, and facilitators), giving feedback and recommendations to PCC managers and PCBH supervisors. No additional auditing is planned.
Data management, monitoring and confidentiality
Data will primarily be collected through and stored in encrypted form in the study technical platform and database (BASS4) provided by Karolinska Core Facilities. Patients and staff fill in questionnaires online. Data are then stored behind two-factor verification and accessible only to selected research personnel. Data collected through means other than self-report forms will be manually entered into BASS4 by research assistants. All data entry is done in premade templates, ensuring that data are in a proper format and within an expected range of values. Any data coding is done according to standardized coding practices. When data collection is finished, all data will be pseudonymized and key coded. Only the principal investigator and university archivists will be able to access the key. Other project team members will only be able to access pseudonymized data. Considering the nature of the trial, which primarily involves data collection from routine clinical practice and presents a low anticipated risk to participants and given that data monitoring for psychological intervention trials is not mandatory in Sweden, we decided to forgo a data monitoring committee due to costs.
Data analysis plan
We are planning to conduct one main analysis for each of the primary outcomes at each level, a range of explorative and longitudinal analyses for the implementation-level outcomes, and several secondary analyses for the organizational- and patient-level outcomes. Primarily, we aim to confirm that the EARLY arm reaches adequate fidelity level faster than the DELAYED arm and to capture the randomized controlled effects of PCBH on organization- and patient-level outcomes when it has reached adequate model fidelity during the first implementation period (corresponding to an implemented-per-protocol perspective). We also want to evaluate the controlled effects of intending to implement PCBH regardless of what level of fidelity each PCC reaches (corresponding to an intent-to-implement perspective) and the non-controlled (i.e., not comparing the randomized groups) relation between reaching adequate model fidelity and outcomes (corresponding to an adherence-effect or dose-response perspective). No interim analyses are planned.
Implemented-per-protocol analyses:
- PCCs in EARLY, from when they have reached FID3, compared to all PCCs in DELAYED, during period 1 (main analysis).
- As (1), but also including data from period 2 in EARLY (but not in DELAYED), to control for the event of PCCs needing extended time to reach FID3, thus lowering the power of the main analysis.
Intent-to-implement analyses:
- All PCCs regardless of model fidelity in EARLY compared to all PCCs in DELAYED during period 1.
- As (3), but for periods 1 and 2.
Adherence-response analyses (not controlled by randomization):
- All PCCs, regardless of arm, when in FID3 during period 1 or 2, compared to PCCs in DELAYED during period 1.
- As (5), but compared to all PCCs before they reach FID3.
Sensitivity Analyses
To control for the lack of an objective cutoff for reaching adequate PCBH, sensitivity analyses of the above tests will be conducted where both FID2 and FID3 are used to define adequate PCBH. In addition, sensitivity analyses will also be performed to manage possible inherent differences in the timing of patient pre-measures. Due to the sometimes fast-paced nature of primary care and the naturalistic study design, patients will at times fill in pre-treatment self-report forms after a first visit rather than before. As such, sensitivity analyses will be conducted to see if the temporal relation between pre-treatment measures and initial visit influences the result.
Statistical tests
Hierarchical Linear Models and Generalized Estimating Equations will be the primary statistical methods used. Missing data will be addressed through appropriate imputation methods that align with the missing data mechanism and account for the multilevel structure of the data. Alternatively, statistical techniques capable of handling missing data, such as multilevel models, will be employed for analysis. The analyses will take into account the multilevel structure of the data, e.g. nesting patients under PCC for patient outcomes. For Health Economic Analyses, effectiveness will be estimated by the construction quality-adjusted life-years (QALYs) based on EQ-5D. Incremental cost-effectiveness ratios (ICERs) will be computed and Monte Carlo simulation with non-parametric bootstrapping will be performed to account for the uncertainty of the ICER point estimates. Cost-effectiveness acceptability curves (CEACs) will be used to explore cost-effectiveness under varying willingness to pay-circumstances.