Study Design
We conducted a 10-week randomized controlled trial (RCT) using a parallel-group design, comparing two active intervention conditions to a waitlist control condition (see Fig. 2). Both intervention conditions had immediate access to the self-help program, and participants in the waitlist control group were given full access ten weeks after randomization.
This trial was conducted and reported following the CONSORT-SPI 2018 checklist [45]. This trial was preregistered with clinicaltrials.gov (NCT04655196, registration date: 07/12/2020), conducted in accordance with the declaration of Helsinki, and has been approved by the Cantonal Ethics Committee Bern (CEC; ID: 202–01298). Moreover, we published a study protocol [46]. Informed consent was obtained from all subjects before participating in the study.
Participants and Procedure
To be included in the study, individuals had to be at least 18 years old, score 18 or higher on the UCLA Loneliness Scale – 9 item version (UCLA-9), have sufficient knowledge of German, have access to the Internet and an Internet-enabled device, and provide a signed consent form and a contact person in case of emergency. Individuals with current severe depressive symptoms, a lifetime diagnosis of psychotic or bipolar disorder, fulfilling the criteria for a current severe substance use disorder, or reporting acute suicidal plans were excluded from the study. Depressive symptoms were assessed with the PHQ-9, and the other exclusion criteria were evaluated with the diagnostic interview [Mini-DIPS-Open Access; 47]. All study participants were allowed to use additional therapeutic services and medication.
Between May 17, 2021, and July 31, 2022, were recruited 243 participants from the general population in German-speaking countries. Participants were recruited via social media, articles/interviews in newspapers, radio interviews, newsletters, google-ads, the study website, and the website listing ongoing studies from our research hub. After registering on the study website and returning a signed informed consent, interested participants received an email link to the baseline assessment. Trained and supervised master- and doctoral students conducted diagnostic interviews [Mini-DIPS-Open Access; 47] with all participants who completed the baseline assessment to assess diagnoses relevant to exclusion from the study. After the diagnostic interview, eligible participants were automatically block-wise randomized with Qualtrics [48] to either the two intervention conditions (Internet-based self-help program with human guidance or automated messages) or the waitlist control group. After the group allocation, participants in the intervention conditions had access to all modules of the Internet-based self-help program. In addition to the baseline assessment, all participants were asked to complete further assessments at 10 weeks (post) after the randomization. After completing the post-assessment, the waitlist control group received access to the intervention in a self-guided format. After randomization, participants and coaches delivering guidance were not blinded concerning the corresponding group allocation. Participants were not compensated for partaking in the trial.
Out of 378 potential participants, 243 met all inclusion and no exclusion criteria and were eligible to participate. In total, 98 participants were randomly assigned to the guidance condition, 97 to the automated message condition, and 48 to the waitlist condition (see Fig. 2). Socio-demographics are reported in Table 1. The sample was mainly female (n = 191, 78.6%), living alone (n = 153, 63.0%), single (n = 182, 74.9%), and had a university degree (n = 151, 62.4%). Participants were between 19 and 80, with a mean age of 45.77 (SD = 14.85) years. A total of 79 (32.5%) were in psychological treatment at baseline, and 125 (51.4%) fulfilled the criteria of at least one psychological disorder according to the diagnostic interview (Mini-DIPS OA). The most prevalent was social anxiety disorder (n = 71, 29.2%). On average, participants experienced loneliness for 11.62 years (n = 238, SD = 13.91, Md = 5.25).
Intervention – SOLUS-D
SOLUS-D. The Internet-based self-help program SOLUS-D is a German adapted version of an Internet-based self-help program developed and pilot-tested in Sweden [29]. The program content is mainly based on cognitive behavioral principles. Compared to the original version, SOLUS-D contains additional modules focusing on mindfulness, self-compassion, and social skills relevant to building or deepening social relationships. SOLUS-D consists of nine modules that are mainly text-based and contain video and audio elements. Each module delivers theoretical information with a specific thematic focus, whereby this content can be deepened and transferred to everyday life with practical exercises. An integrated diary function was additionally aimed at changing the attentional focus and becoming more aware of compassion for the self and others in everyday life. A detailed description of the program content can be found in Supplementary Table S1. We recommended working on one module per week, corresponding to an approximate weekly time commitment of 50 minutes. However, participants could spend more time on the modules, corresponding exercises, and diaries. As the modules build on each other, we recommended working on them in a specific sequence. However, as the order of the program content might not suit everyone, all modules were unlocked from the beginning rather than every week. Participants were free to repeat content and exercises upon their preferences. The program was accessible by computer, smartphone, or tablet. Secure Socket Layer encryption was used to secure Internet-based communication with the program and the guides. Within the program, participants were only identifiable with anonymous login names, and they had a personal, password-protected login for the program.
Study Conditions
Participants in the “Guidance”- condition (GU) had access to SOLUS-D one day after randomization. They received weekly individualized feedback (i.e., guidance) from trained and supervised coaches through the message function of the self-help program. Participants were informed via the study information and after group allocation that a coach sent the weekly messages. The messages entailed feedback on participants' work within the program during the previous week and answered individual questions. The primary aim of the guidance was to motivate participants to continue with the program. The main content of the messages was semi-structured and manualized according to the theoretical model of Supportive Accountability [49]. This model aims to increase adherence through human contact by being accountable to a coach. The coaches sent participants who did not log into the program in the previous week a standardized reminder. Reminders were sent for up to three consecutive weeks if participants did not log into the program or react to the reminders. The coaches were two psychologists with a master’s degree in their first year of a CBT post-graduate program and ten master’s students in their last term of a graduate program in clinical psychology. The authors NS, AS, and TK trained and supervised the coaches. On average, the coaches spent 17.10 minutes (SD = 10.15, Md = 14.25) on guidance per participant per week.
Participants in the “Automated Message”-condition (AM) had access to SOLUS-D one day after randomization and received weekly standardized messages via email. Participants were informed via the study information and after group allocation that the weekly messages were sent automatically and not by a study team member, i.e., a human being. The automated messages aimed to motivate participants to continue working with the program, e.g., by summarizing the module contents of the previous week and providing an outlook of the next module. After receiving access to the intervention, participants in the AM conditions were informed that upcoming technical problems could be addressed to the study team.
Participants in the “Waitlist Control Group” (WL) received access to the intervention in a unguided format, i.e., without guidance or automated messages, ten weeks after randomization upon completing the post-assessment. After receiving access to the intervention, participants in the WL condition were informed that upcoming technical problems or questions regarding the program could be addressed to the study team.
Measures
Demographic variables (e.g., gender, age, and education level) and therapy and medication status were self-reported by the participants at baseline. Self-reported primary and secondary outcome measures were assessed at baseline, and 10 weeks after randomization. Participants who did not respond to the assessment invitation received up to three weekly reminders via email. All questionnaires were administered in German and completed on the online survey platform Qualtrics [48] by the participants. The diagnostic interview was administered via telephone.
Primary Outcome
Loneliness, measured at the post-assessment timepoint, was the primary outcome and was assessed with the 9-item short version (UCLA-9) [50] of the UCLA Loneliness Scale [51, 52]. The original scale consists of 20 Items and assesses three dimensions of loneliness: intimate, relational, and collective. The nine-item version consists of the three items with the highest factor loadings on each facet of loneliness [53]. The validity and reliability of the short version are comparable with those of the 20-item original scale [54]. The response options are (1) never, (2) rarely, (3) sometimes, and (4) always. After recoding reverse-coded items, all items are summed up, and the total score ranges from 9 to 36, with higher values indicating more pronounced feelings of loneliness. Cronbach’s α for the UCLA-9 at post-assessment was 0.83. Internal consistency at post-assessment is reported since baseline data were affected by range restriction and biased reliability since we used the UCLA-9 as an inclusion criterion [55]. As previous studies detected differences, e.g., in the prevalence of loneliness, depending on either directly (i.e., using the word “lonely”) or indirectly (i.e., not mentioning the word “lonely”) measuring loneliness [41], an additional single item was administered to assess loneliness directly. Furthermore, an additional 3-item short form of the UCLA Loneliness Scale (UCLA-3) [56] was used, as norms for the German population exist [37].
Secondary Outcomes
Depressive symptoms were assessed with the 9-item depression module of the Patient Health Questionnaire (PHQ-9) [57, 58]. The short form of the Social Interaction Anxiety and Social Phobia Scale (SIAS-6 & SPS-6) [59] was used to assess symptoms of social anxiety. Satisfaction with life was measured with the 5-item Satisfaction with Life Scale (SWLS) [60, 61]. Furthermore, we assessed self-esteem with the 10-item revised German version [62] of the Rosenberg Self-Esteem Scale (RSES) [63] and used the 20-item Sussex-Oxford Compassion for the Self Scale (SOCS-S) [64] to measure self-compassion. The Social Network Index (SNI) [65] was administered to assess objective social isolation, i.e., network size [66]. We used the Personality Inventory for the DSM-5 Brief Form Plus (PID5BF+) [67] to assess maladaptive personality traits. Interpretation bias was assessed with the respective subscale of the Interpretation and Judgmental Bias Questionnaire (IJQ) [68, 69]. The Adult-Rejection Sensitivity Questionnaire (A-RSQ) [70] was used to measure rejection sensitivity. Furthermore, we administered the subscale Behavior-social avoidance of the Cognitive-Behavioral Avoidance Scale (CBAS) [71, 72] to assess social avoidance behavior. The Distress Disclosure Index (DDI) [73] measured comfort with self-disclosure. We administered the Kernis Goldman Authenticity Inventory - short form (KGAI-SF) [74] to assess authenticity. We used the corresponding subscale of the Bern Embitterment Inventory (BVI) [75] to assess misanthropy. Self-determined motivation for solitude was assessed with the respective subscale from the Motivation for Solitude Scale – Short Form (MSS-SF) [76].
Further measures
At post-assessment, participants in both intervention groups completed measures on client satisfaction (CSQ-8) [77] and usability (SUS) [38] of the intervention. Moreover, we assessed negative effects that occurred during the intervention phase and were attributed to the intervention by participants in the intervention conditions with the INEP [78] at post-assessment. Adherence to the Internet-based program was assessed as the number of modules completed. A module was considered completed when each page per module had been clicked at least once. Furthermore, the time participants spent within the program was measured. The coaches noted down the amount of time they spent reading the participants’ content within the program, as well as writing and delivering guidance. Before randomization, we administered the Mini-DIPS-Open Access [47] to assess diagnoses of mental disorders. We refer to the online Supplementary Material and the study protocol [46] for a more detailed description of all measures.
Statistical Analyses
Following the intention-to-treat principle (ITT), we included all randomized participants in the primary analyses. We computed ANOVAs for continuous and Chi-Square tests for nominal data to assess group differences at baseline and group comparisons regarding reliable change. Independent sample t-tests were performed to determine group differences in program usage, satisfaction with the program, and negative effects due to the intervention. Where relevant assumptions for the respective tests were violated, we conducted non-parametric tests, e.g., Fisher’s Exact Test or Kruskal-Wallis Test. For the primary analyses, we used linear mixed models with restricted information maximum likelihood estimation in the lme4 package [79] in R (version 4.2.1) to evaluate change in the primary and secondary outcome variables. Linear mixed models are suitable for longitudinal data with repeated measures, as the dependency of the data is accounted for [80]. Furthermore, linear mixed models yield robust estimates despite missing data, accounting for it through maximum likelihood estimation, which produces unbiased estimates under the Missing at Random (MAR) assumption [81]. We estimated linear mixed models for the primary and each secondary outcome separately with fixed effects of time, condition, Time × Group interaction, and random intercepts for participants to evaluate the efficacy of the intervention. Time and condition were entered into the models as categorical variables. We did not include random slopes as the convergence of the model could not be achieved. Significant Time × Group interactions were followed up with planned contrast analyses, where we compared the two intervention conditions against the waitlist condition (GU: -0.5, AM = -0.5, WL = 1) and the two intervention conditions against each other (GU: -1, AM = 1, WL = 0). Following Feingold [82], between-group effect sizes (i.e., Cohen’s d) were calculated by dividing the estimated mean difference at post-assessment by the pooled standard deviation at baseline. Within-group effect sizes were calculated by dividing the difference between the estimated means (pre-post) by the pooled standard deviation of the observed means from both time points. Additionally, we estimated 95% confidence intervals for the effect sizes. The α error level was set to .05. Only the primary outcome measure and the PHQ-9 required participants to answer all items. This was not the case for the other questionnaires to reduce the attrition rate. Accordingly, for scales with missing values at the item level, the scale scores were calculated with the available data [83].
Reliable improvement or deterioration in the primary outcome was calculated using the reliable change index (RCI) [84]. To determine the reliable change index, we used Cronbach’s alpha (.90) of the UCLA-9 from a sample of the general population of German-speaking countries (n = 813, unpublished data) and the current study samples' standard deviation at baseline (SD = 3.34). Participants with change scores (pre-post) greater than 2.93 on the UCLA-9 were classified as reliably improved, not changed when scoring between 2.93 and − 2.93, and deteriorated with a change score lower than − 2.93. To ensure a conservative estimate of the change in loneliness, reliable change was computed using the ITT sample, replacing missing values at post-assessment with the last observation carried forward. Additionally, reliable change was calculated in the per-protocol sample consisting of participants who completed the baseline- and post-assessment and logged into four or more modules (i.e., minimal therapeutic contact).
High dropout rates in studies on Internet-based self-help programs are common and can lead to biased results. To check the robustness of the results, we additionally conducted sensitivity analyses and ran the primary analyses with the per-protocol sample. We conducted further sensitivity analyses focusing on different subgroups, i.e., participants fulfilling at least one psychological disorder and participants indicating to attend psychotherapeutic treatment at baseline.
Sample Size and Power
We conducted an a priori power analysis using G*Power 3 [85] and aimed at detecting small effect sizes [86] of f = .10 (equivalent to Cohen d = 0.20) for the Time × Group interaction for the two intervention conditions at an α error level of .05., a power (1-β) of 0.80, and with correlations of r = .60 between pre-and post-treatment measures, as found in a previously conducted trial on ICBT for loneliness [29]. According to the power analysis, a sample size of 80 participants per intervention group was sufficient to detect statistically significant differences with these assumptions. Furthermore, to account for dropouts of approximately 25%, we decided to randomize 100 participants to each intervention group. Concerning the comparison between the intervention and waitlist control groups, 50 participants were considered sufficient for the waitlist since between-group effects were expected to be medium-to-large, based on the Swedish trials mentioned above [29, 30]. Thus, we intended to randomize 250 participants (randomization ratio: 2:2:1). For regulatory reasons, we had to end recruitment when 243 participants were randomized, which might have limited our ability to detect the intended effects.