Demographic and clinical characteristics associated with anxiety and depressive symptom outcomes in users of a digital mental health intervention incorporating a relational agent

doi:10.21203/rs.3.rs-2488688/v1

Background

Digital mental health interventions (DMHIs) may reduce treatment access issues for those experiencing depressive and/or anxiety symptoms. DMHIs that incorporate relational agents may offer unique ways to engage and respond to users and to potentially help reduce provider burden. This study tested WB-LIFE, a DMHI that employs Woebot, a relational agent that delivers evidence-based psychotherapies, among those with baseline clinical levels of depressive or anxiety symptoms. Self-reported depressive and anxiety symptom changes between baseline and end of 8-week intervention, as well as the association between demographic and clinical characteristics and each outcome, were measured

Methods

This exploratory, single-arm, 8-week study of 256 adults yielded non-mutually exclusive subsamples with either clinical levels of depressive or anxiety symptoms at baseline. Week 8 Patient Health Questionnaire-8 (PHQ-8) changes were measured in the depressive subsample (PHQ-8≥10). Week 8 Generalized Anxiety Disorder-7 (GAD-7) changes were measured in the anxiety subsample(GAD-7≥10). Demographic and clinical characteristics were examined in association with symptom changes via bivariate and multiple regression models adjusted for WB-LIFE utilization. Characteristics included age, sex at birth, race/ethnicity, marital status, education, sexual orientation, employment status, health insurance, baseline levels of depressive and anxiety symptoms, and concurrent psychotherapeutic or psychotropic medication treatments during the study.

Results

Both the depressive and anxiety subsamples were predominantly female, educated, non-Hispanic white, and averaged 38 and 37 years of age, respectively. The depressive subsample had significant reductions in depressive symptoms at Week 8 (mean change= - 7.28, p<0.01); the anxiety subsample had significant reductions in anxiety symptoms at Week 8 (mean change= -7.45, p<0.01). No significant associations were found between sex at birth, age, employment status, educational background and Week 8 symptom changes. Significant associations between depressive and anxiety symptom outcomes and sexual orientation, marital status, concurrent mental health treatment, and baseline symptom severity were found.

Conclusions

The present study suggests early promise for WB-LIFE as an intervention for depression and/or anxiety symptoms. Although exploratory in nature, this study revealed potential user characteristics associated with outcomes that can be investigated in future studies.

Trial Registration

This study was retrospectively registered on ClinicalTrials.gov (#NCT05672745) on 05/01/2023.

digital mental health intervention

chatbot

conversational agent

relational agent

depression

anxiety

user characteristics

natural language processing

artificial intelligence

Mental health issues have consistently contributed to the worldwide disease burden [1]. This held true before but was exacerbated by the COVID-19 pandemic, given its associated increase of 25% in depressive and anxiety disorders [2]. Stressing already limited mental health treatment access and provider capacity, the increased prevalence has left even more individuals without needed care [3,4]. In order to mitigate such prolific access and capacity issues, people experiencing symptoms of depression and anxiety may seek interventions with alternative delivery modalities [5]. Indeed, the heightened need for treatment access has led to an increase in the use and study of technology-delivered mental health interventions [6]. Such interventions, known as digital mental health interventions (DMHIs), offer self-guided (no human support), guided (human support specific to the DMHI offered via telephone, video, or text), or blended (a combination of DMHI plus human support) approaches to deliver treatment modules and skill-building interventions [7]. DMHIs have included therapeutic techniques from one or more evidence-based psychotherapies, such as cognitive behavioral therapy (CBT) [7]. In aggregate, DMHIs have been shown to produce at least moderate improvements in depressive and anxiety symptoms [8–10].

Among DMHIs, there has been increased attention on and proliferation of mental health chatbots [11,12] also known as conversational agents [13,14]. Conversational agent-delivered interventions are often available on a smartphone application (app), enabling users access at any time for support and to address immediate problems contextually in real time. Like the broader DMHI category, conversational agent-delivered interventions may be founded in evidence-based psychotherapeutic approaches such as CBT, Acceptance and Commitment Therapy (ACT), and Dialectical Behavior Therapy (DBT) [7,15–17]. In addition to thoughtful conversational design, some conversational agents are bolstered by artificial intelligence and/or natural language processing (NLP) [18], which may enable response by delivery of therapeutic intervention and psychoeducational content. Conversational agent-delivered interventions may be particularly useful for those who wish to receive help without speaking to a clinician due to stigma or cost concerns [13]. Recent meta-analyses of conversational agent delivered DMHIs indicate that they are generally viewed favorably by users and may enhance access to mental health care and improve mental health outcomes [11,13,19].

When conversational agents are built to form an alliance with users over time, they may be referred to as relational agents [20,21]. Some data have demonstrated that relational agents are able to form a working therapeutic alliance with users [22], which has been linked to improved mental health outcomes[23]. These capabilities, coupled with their immediate availability in someone’s moment of need, offer the potential for relational agents to extend the current menu of mental health services available to those in need. Unfortunately, little is known about the characteristics of users who may benefit from relational agent-delivered DMHI [11].

The broader, non-relational agent-specific DMHI literature provides some clues about what types of user demographic or clinical characteristics might be associated with outcomes among relational agent users, but much of the evidence has been inconclusive. DMHI studies of depression and anxiety have reported inconsistent findings in outcomes by characteristics such as gender, age, marital status, or education [24–29]. Although there is preliminary evidence of the feasibility and acceptability of DMHI interventions for racial/ethnic, LGBTQ (lesbian, gay, bisexual, transgender, queer or questioning), and other marginalized or underserved groups, these demographic and self-identity characteristics have been underexplored as predictors of outcomes in the DMHI literature [30,31]. Individual DMHI studies addressing racial and ethnic minorities have shown promise in reducing depressive symptoms, but large scale prospective studies are lacking [31].

With regard to clinical characteristics, non-relational agent-specific DMHI studies have generally found that users with higher severity levels of anxiety and depression tend to achieve greater improvement in these symptoms than those with lesser severity [8,10,32], likely an artifact that these users have more room for improvement [8]. Surprisingly, the association between concurrent mental health treatment (i.e., psychotherapy and/or psychotropic medications) and DMHI-related changes in depression and anxiety symptoms has seen little investigation. One study of health care workers during the COVID-19 pandemic found that exposure to a mobile phone–based mental health intervention did not significantly affect Depression Anxiety Stress Scale-21 (DASS-21) scores in the overall sample, but a subgroup analysis of those receiving psychotherapy or psychotropic medications found significant improvements in this group [33]. While some data exist on demographic and clinical characteristic associations with mental health outcomes among the broad category of DMHIs, the authors could find no such data in the literature specific to relational agent-delivered DMHIs. Thus, further exploration of putative demographic and clinical characteristics among users of a relational agent-delivered DMHI is needed to illuminate optimal uses as well as perhaps tailor interventions to the unique needs of individuals experiencing anxiety and depressive symptoms.

This present study evaluated WB-LIFE, an intervention delivered via Woebot, a relational agent that utilizes thoughtful conversational design and some NLP to deliver CBT as well as elements of Interpersonal Psychotherapy (IPT) and DBT through a text-based interface on a smartphone app. Feasibility, acceptability, and efficacy studies with various populations using Woebot-based interventions are promising. These studies include young adults reporting depressive symptoms [34], adults with substance abuse issues [35], adolescents with depressive and/or anxiety symptoms [36], and adult women with postpartum mood concerns [37,38].

The present study sought to address gaps in the literature specific to a relational agent-delivered DMHI by first ascertaining the degree of depressive or anxiety symptom changes after 8 weeks of WB-LIFE use among participants with self-reported clinical levels of depressive or anxiety symptoms at baseline, respectively. In addition, the study explored associations between participant demographic and clinical characteristics and depressive and anxiety symptom outcomes. The research questions in this exploratory study included the following:

1. Do participants with clinically elevated depressive symptoms at baseline have significant decreases in symptoms at Week 8?

2. Do participants with clinically elevated anxiety symptoms at baseline have significant decreases in symptoms at Week 8?

3. Does the amount of reduction in depressive and anxiety symptoms respectively, at Week 8, differ by demographic characteristics, clinical characteristics (i.e., baseline anxiety and depression symptom severity), and involvement in concurrent mental health treatment?

Overall study description

This exploratory study was non-randomized, single-armed, and open-labeled. Study procedures were approved by the WIRB-Copernicus IRB Group (WCG) institutional review board on January 20, 2022. All participants provided informed consent. Data was collected from WB-LIFE participants between May 11, 2022 and July 20, 2022. The study was retrospectively registered on ClinicalTrials.gov (#NCT05672745) on 05/01/2023.

Recruitment

A convenience sample was recruited via social media advertisements, such as Facebook and Instagram, containing an embedded hyperlink to the study’s electronic consent form. Upon signing the consent form, study staff emailed participants a link to complete the self-report screening (assessing inclusion and exclusion criteria) and baseline survey questions. The inclusion criteria were: (1) 18 years of age or older, (2) residence in the United States (U.S.), (3) reported English literacy, (4) availability to complete study activities, and (5) ownership or regular access to a smartphone with WiFi and sufficient data access for the duration of the study. The exclusion criteria were: (1) current suicidal ideation with a plan or intent to act or a suicide attempt within the past 12 months, (2) a reported lifetime diagnosis of a psychotic disorder (including schizophrenia or schizoaffective disorder) or bipolar disorder, or (3) previous use of a Woebot application.

Participants who met inclusion and exclusion criteria and who completed the baseline survey assessments within 5 days of receipt were immediately provided with a link to access and download WB-LIFE. They were instructed to complete registration within 3 days using the study’s referral code and the same credentials entered at consent. Those who failed to register within 3 days became ineligible for the study. Participants who did not meet eligibility criteria were thanked for their interest, given a list of national crisis resources, and exited from the study.

Study Procedures

Enrolled participants entered the 8-week intervention period, where they were invited to engage with WB-LIFE. Participants completed survey assessments that were delivered outside of the application via email invitation at Day 3, Week 4 (mid-intervention), and Week 8 (end of intervention; EOI). Upon the completion of the final survey, participants exited the study and their access to WB-LIFE was discontinued. Participants received a maximum of $100 in Amazon gift cards for completing study assessments ($20, $30, and $50 for Day 3, Week 4, and Week 8, respectively). This study reports data collected at baseline and Week 8 (EOI). Fig.1 describes the participant flow through the study.

WB-LIFE Intervention

WB-LIFE is accessible as an iOS or Android smartphone application and hosts the relational agent Woebot to guide users through psychotherapeutic content delivered via text-based messages. Using proprietary natural language processing, the user experience centers around goal-oriented conversations based on self-reported real-time needs. These conversations are tailored to help the user develop emotion regulation skills in the context of problems experienced in their everyday lives. WB-LIFE is grounded in scientifically validated psychotherapies (e.g., CBT, IPT, DBT) and helps users address mood monitoring and management, as well as utilizes tools such as progress reflection, gratitude journaling, and mindfulness practice. Previous studies of various Woebot applications have provided evidence of feasibility, acceptability and efficacy [34,35,38,39], as well as demonstrated an ability to form a working alliance with users [22]. Sample screenshots from WB-LIFE are shown in Fig. 2.

Safety

WB-LIFE is not a crisis service. However, it is equipped with an NLP algorithm that detects patterns in user free-text input that may suggest potential concerning language. Upon detection, WB-LIFE asks the user to confirm if they are in crisis. For all confirmations, WB-LIFE provides a list of resources that includes emergency contact phone numbers and suicide crisis hotline information. WB-LIFE then offers tools to assist with addressing upsetting emotions and thoughts in the moment. The list of resources is also immediately available at any time to the user either from the settings menu or by entering “SOS” in free text. Potential concerning language detection triggers and confirmations were not actively monitored by staff during the study.

Due to the low-risk profile of this study, adverse events (AEs) were assessed by spontaneous participant reports communicated outside of WB-LIFE and followed a predetermined safety protocol. In the event of a spontaneously reported AE to any study staff, the Principal Investigator would be immediately alerted with all available clinical and other details of the event. Based on this initial report, appropriate steps would be taken to gather further information as needed and address the AE. During this study, no such AEs were reported.

Assessments

Participants were emailed links to survey assessments through the Castor Electronic Data Capture (EDC) [40] platform, with a 7-day window to complete the baseline and Week 8 surveys. We tracked the assessment completion rate, defined as completion of both the PHQ-8 and GAD-7 as baseline, Day 3, Week 4, and Week 8.

Demographic Battery

The baseline survey assessed demographic characteristics, including age, race/ethnicity, marital

status, education, sexual orientation, employment status, and health insurance. Gender identity, while collected, did not yield enough individuals who identified outside of the most common categories (Man or Woman) to allow for inclusion in the models. Instead, we included sex assigned at birth.

Clinical Characteristics

Concurrent Mental Health Treatment. Concurrent treatment was defined as mental health treatment at any point in the study. At baseline, participants were asked to indicate if they were currently seeing a therapist for mental health concerns and/or if they were taking medications for a psychiatric condition. At the Week 8 assessment, participants were asked if psychotropic medications and/or psychotherapy were continued or if either was started during the course of the 8-week study intervention.

Patient Health Questionnaire (PHQ-8). The PHQ-8 is a widely used 8-item self-report measure, with demonstrated reliability and sensitivity to clinical change. It is used to screen for depression, measure symptom severity, and assess outcome [41]. Respondents are asked to consider symptoms over the past two weeks and to indicate frequency of symptoms from not at all (0) to nearly every day (3). The total sum scores range from 0-24, with the following severity cutoffs - minimal (0-4), mild (5-9), moderate (10-14), moderately severe (15-19), and severe (20-24). A score of 10 or greater is considered an acceptable clinical cutoff for major depression, having good sensitivity and specificity [42]. PHQ-8 scores at baseline and Week 8 were analyzed in this study.

Generalized Anxiety Disorder (GAD-7). The GAD-7 is a brief self-report tool to assess the frequency and severity of anxious thoughts and behaviors [43]. Respondents are asked to indicate frequency of symptoms from not at all (0) to nearly every day (3) over the previous two weeks. Total sum scores range from 0-21, with the following severity cutoffs - minimal (0-4), mild (5-9), moderate (10-14), and severe (15-21). A score of 10 or greater is considered a reasonable clinical cutoff for generalized anxiety disorder [43].

WB-LIFE Utilization

Over the course of the study, we tracked participant WB-LIFE log-ins. Completion of the

intervention was defined as logging in to WB-LIFE during at least 50% (i.e., at least 4 of 8) of the study weeks (yes vs. no). The per protocol (PP) population was defined as having completed both the intervention and the 8-week (PHQ-8 and GAD-7) study assessments.

Statistical Analysis

Each set of analyses was performed on the two analytic subsamples: study participants with PHQ-8 ≥ 10 at baseline and those with GAD-7 ≥ 10 at baseline. Participants with clinically elevated symptoms of both depressive and anxiety symptoms were included in both subsamples (although multiple regression models adjusted for the presence of the other symptoms in analyses). Continuous demographic and clinical variables were summarized by means and standard deviations; discrete and/or ordinal variables were summarized by reporting the sample size and proportion of participants in each category.

To address the first research question, a paired t-test was applied to the baseline and Week 8 PHQ-8 scores for the subgroup defined as having a baseline PHQ-8 ≥ 10. Similarly, for those with elevated anxiety symptoms at baseline (GAD-7 ≥ 10), a paired t-test was applied to the baseline and Week 8 GAD-7 scores to address the second research question. For both instances, the tests were intent-to-treat and two-sided, with alpha=0.05.

To assess the third research question, sets of bivariate and multiple linear regression models were fit among those with PHQ-8 ≥ 10 and also among those with GAD-7 ≥ 10 in separate sets of models. Each regression model included change in depressive or anxiety symptom scores as outcomes, respectively, and each demographic and clinical characteristic as independent variables. First, bivariate regressions were run between each outcome and each demographic and clinical characteristic, separately. Then, a multiple regression model that included all demographic and clinical characteristics as well as an indicator of utilization to adjust for WB-LIFE use (dichotomous variable of having any WB-LIFE use in at least 50%, or 4 of 8, weeks of the study) were run and, using Akakie’s Information Criterion (AIC) for model selection, rerun until the final model was attained. Final models also removed all variables with Variance Inflation Factors greater than 4 to account for multicollinearity. Sensitivity analyses were performed with the PP population, which included participants who used the app in at least 4 of 8 study weeks and who completed the end of study (EOS) PHQ-8 and GAD-7 assessments; the same modeling approach used for the intention to treat (ITT) population was applied to the PP population. The level of significance for all final models was 0.05. All analyses were performed with the R statistical software program [44].

Sample

As detailed in the CONSORT diagram (Fig. 3), 1105 potential participants were screened. Of these, 485 were excluded for not meeting eligibility criteria. After enrollment, an additional 358 participants with unauthorized accounts (e.g., duplicate registrants) were withdrawn from the study and excluded from the analyses. An additional six unauthorized registrants completed study procedures and were subsequently removed from the analytic sample, given that they were initially missed as a result of a clerical error. Thus, the final sample included in analyses consisted of 256 participants. Of these 256 participants, 245 completed the Day 3 survey, 236 completed the Week 4 surveys, and 234 completed the Week 8 surveys.

CONSORT diagram. Although Day 3 and Week 4 survey data were collected as part of the parent study, they were not part of the analyses included in the present report.

Sample Characteristics

The analytic subsamples included 111 participants with elevated baseline levels of depressive symptoms and 107 participants with elevated baseline levels of anxiety symptoms at baseline as defined by a PHQ-8 score ≥ 10 and/or GAD-7 score ≥ 10, respectively. The sample size for the combined (elevated on PHQ-8 or GAD-7) groups comprised 139 participants. Some participants with elevated symptoms of both depressive and anxiety symptoms were in both analytic subsamples (n=79). Both the depressive symptom and anxiety symptom samples were predominantly female, educated, and employed, identified as non-Hispanic white and heterosexual, and had private health insurance (Table 1) with mean ages of 38 and 37 years, respectively. Among those with clinical levels of depressive symptoms, 50% had symptoms falling in the moderately severe or severe range (PHQ-8 score ≥ 15) and 49% reported concurrent mental health treatment during the study. Among those with clinical levels of anxiety symptoms, 44% had symptoms falling in the severe range (GAD-7 score ≥ 15) and 53% reported concurrent mental health treatment during the study.

The assessment completion rate at Week 8 was 91%. The PP population examined in sensitivity analyses comprised 177 (69%) of the 256 participants in the analytic sample after removing 67 (26%) who used WB-LIFE for at most 3 of 8 weeks and removing 22 (9%) whose EOS PHQ-8 and GAD-7 assessments at Week 8 were fully missing (no item scores were completed for each measure); the removals were not mutually exclusive. Sixty-four percent (N=71/111) of the elevated depressive symptoms subgroup met per protocol criteria and 59% (N=63/107) of the elevated anxiety symptoms subgroup met these criteria.

Table 1. Baseline sociodemographic and clinical characteristics, adherence, and satisfaction among participants with clinical levels of baseline depressive or anxiety symptoms

Participants with PHQ-8 baseline score ≥ 10

N= 111

n (%)

Participants with GAD-7 baseline score ≥ 10

N= 107

n (%)

Sociodemographic Characteristic

Age in years (mean, sd)

38 (13.57)

37 (12.28)

Race/ethnicity

Non-Hispanic Black

Non-Hispanic White

Other

37 (33%)

55 (50%)

19 (17%)

32 (30%)

56 (52%)

19 (18%)

Sex at Birth

Female

Male

83 (75%)

28 (25%)

81 (76%)

26 (24%)

Sexual Orientation

Sexual Minority

Heterosexual

19 (17%)

92 (83%)

19 (18%)

88 (82%)

Education

Graduate or Postgraduate Degree

College Degree

Some College or technical school

High School (Grades 9 - 12)

32 (29%)

43 (39%)

21 (19%)

14 (13%)

24 (23%)

43 (41%)

22 (21%)

16 (15%)

Employment

Full Time Employed

Part Time Employed

Not Employed

Other

53 (49%)

11 (10%)

29 (27%)

15 (14%)

57 (54%)

9 (9%)

22 (21%)

17 (16%)

Marital Status

Divorced/Separated/Widowed

Married/Partnered/cohabiting

Single

10 (9%)

50 (47%)

47 (44%)

10 (10%)

48 (46%)

47 (45%)

Health Insurance

Government Based Insurance

Private Insurance

No Insurance/ Prefer not to answer

40 (37%)

46 (43%)

21 (20%)

37 (36%)

45 (43%)

22 (21%)

Clinical Characteristics

Baseline Depressive symptom severity

Minimal

Mild

Moderate

Moderate-severe

Severe

0 (0%)

55 (50%)

30 (27%)

26 (23%)

7 (7%)

21 (20%)

25 (23%)

29 (27%)

25 (23%)

Baseline Anxiety symptom severity

Minimal

Mild

Moderate

Severe

3 (3%)

29 (26%)

37 (33%)

42 (38%)

0 (0%)

60 (56%)

47 (44%)

Concurrent Mental Health Treatment*

Any (Concurrent treatment)

None (Woebot only)

54 (49%)

57 (51%)

57 (53%)

50 (47%)

Adherence Metrics

App Utilization

Use of WB-LIFE in at least 4 of 8 weeks

74 (67%)

69 (64%)

Per protocol data set

71 (64%)

63 (59%)

GAD-7 = Generalized Anxiety DIsorder-7 item scale; PHQ-8 = Patient Health Questionnaire-8 item scale

* Concurrent mental health treatment = any psychotherapy or psychotropic medication use at any time during the study

“Per protocol” was defined as using WB-LIFE in at least 4 of 8 study weeks and completing end of study PHQ-8 and GAD-7 assessments.

Depressive Symptoms

In the subgroup with clinically elevated depressive symptoms, PHQ-8 scores significantly decreased over the 8-week study period (mean change -7.28, p < .001). In bivariate regression models, the amount of symptom decline did not significantly differ by age, educational level, employment, or baseline level of anxiety symptoms but did significantly differ across several characteristics tested (Table 2). Non-Hispanic Blacks, those who were single, and those with severe levels of baseline depressive symptoms demonstrated significantly greater amounts of decline in depressive symptoms than Non-Hispanic Whites, those who were married/partnered/cohabiting, and those with moderate levels of baseline depressive symptoms, respectively. Females, participants identifying as sexual minorities, those with private or government-based health insurance, those who used the program on at least 4 of the 8 weeks, and those in concurrent mental health treatment demonstrated significantly lower amounts of decline in depressive symptoms than males, participants identifying as heterosexuals, those with no health insurance or who declined to answer the health insurance question, those who used the program on less than 4 of the 8 weeks, and those not in concurrent mental health treatment, respectively. PP analyses largely confirmed the ITT findings (Supplementary Table 1).

After removing race/ethnicity, insurance, and employment from consideration because of multicollinearity, the final multiple regression model indicated significant differences in the amount of decrease in depressive symptoms at Week 8 by sexual orientation, marital status, baseline depressive symptom severity category, and concurrent mental health treatment (Table 3) in the same directions as each bivariate model described above (in addition, divorced individuals had greater amounts of depressive symptom declines as compared with married/partnered, cohabiting).

Table 2. Unadjusted Bivariate Linear Regression Models of Characteristics Associated with change scores from baseline to Week 8 in depressive symptoms among those with clinically elevated levels of baseline depressive symptoms: PHQ-8 ≥ 10

	PHQ-8: Week 8 Change Scores
Characteristics of Interest	Estimates	95% CI	p-value
Use of WB-LIFE in at least 4 of 8 weeks	2.9	0.51, 5.4	0.018
Age	0.03	-0.05, 0.12	0.4
Race/Ethnicity Non-Hispanic White	Reference Level
Non-Hispanic Black	-6.0	-8.3, -3.7	<0.001
Other	0.02	-2.8, 2.9	>0.9
Sex at Birth Male	Reference Level
Female	2.7	0.09, 5.3	0.042
Sexual Orientation: Heterosexual	Reference Level
Sexual Minority	5.6	2.6, 8.6	<0.001
Education High School	Reference Level
College Degree	-1.9	-5.6, 1.8	0.3
Graduate or postgraduate degree	-0.68	-4.4, 3.1	0.7
Some college or technical school	1.1	-3.1, 5.3	0.6
Employment Full Time	Reference Level
Not Employed	2.4	-0.36, 5.2	0.087
Other	3.1	-0.49, 6.8	0.089
Part Time Employed	3.6	-0.32, 7.4	0.071
Marital Status Married/Partnered/Cohabiting	Reference Level
Divorced/Separated/Widowed	-2.2	-6.2, 1.7	0.3
Single	-4.1	-6.6, -1.7	<0.001
Health Insurance: No Insurance/Prefer not to answer	Reference Level
Government based Insurance	6.9	4.2, 9.7	<0.001
Private Insurance	7.5	4.8, 10	<0.001
BL Depressive Symptom Severity Moderate	Reference Level
Moderate-Severe	-1.1	-3.2, 1.0	0.3
Severe	-9.3	-12, -7.0	<0.001
BL Anxiety Symptom Severity Minimal	Reference Level
Mild	-1.6	-9.5, 6.2	0.7
Moderate	-2.1	-9.9, 5.7	0.6
Severe	-7.1	-15, 0.71	0.074
Concurrent Mental Health Treatment*	3.2	0.94, 5.4	0.006

BL = baseline; PHQ-8 = Patient Health Questionnaire-8 item scale

* Concurrent mental health treatment = any psychotherapy or psychotropic medication use at any time during the study

Table 3. Adjusted Linear Regression Model (Multiple Regression) of change scores from baseline to Week 8 in depressive symptoms among those with clinically elevated levels of baseline depressive symptoms: PHQ-8 ≥ 10

	PHQ-8: Week 8 Change Scores
Characteristics of Interest	Estimates	95% CI	p-value
(Intercept)	-2.46	-10.16 – 5.24	0.527
Used the app in at least 4 of 8 weeks	0.2	-1.75 – 2.15	0.84
Age	0.05	-0.03 – 0.14	0.229
Sex at Birth Male	Reference Level
Female	-0.96	-3.24 – 1.32	0.405
Sexual Orientation Heterosexual	Reference Level
Sexual Minority	5.71	2.88 – 8.54	<0.001
Education High School	Reference Level
College Degree	-1.63	-4.33 – 1.07	0.234
Graduate or postgraduate degree	-0.24	-3.18 – 2.70	0.873
Some college or technical school	0.88	-2.26 – 4.01	0.58
Marital Status Married/Partnered/Cohabiting	Reference Level
Divorced/Separated/Widowed	-3.86	-7.38 – -0.34	0.032
Single	-3.75	-5.78 – -1.72	<0.001
BL Depressive Symptom Severity Moderate	Reference Level
Moderate-Severe	-0.09	-2.44 – 2.25	0.937
Severe	-5.32	-8.16 – -2.47	<0.001
BL Anxiety Symptom Severity Minimal	Reference Level
Mild	-3.97	-10.29 – 2.35	0.215
Moderate	-4.5	-11.02 – 2.02	0.173
Severe	-5.59	-12.17 – 0.99	0.095
Concurrent Mental Health Treatment*	2.62	0.80 – 4.43	0.005

BL = baseline; PHQ-8 = Patient Health Questionnaire-8 item scale

* Concurrent mental health treatment = any psychotherapy or psychotropic medication use at any time during the study

NOTE: Final models did not consider race/ethnicity, employment, or insurance status because of multicollinearity.

Anxiety Symptoms

Average GAD-7 scores for each subgroup of interest significantly improved over the course of the study. In the subgroup with clinically elevated anxiety symptoms at baseline, GAD-7 scores significantly decreased over the 8-week study period (mean change -7.45, p < .001). The amount of symptom decline did not significantly differ by use of WB-LIFE at least 4 of 8 weeks, age, sex at birth, educational level, or employment, but did significantly differ across several characteristics tested (Table 4). Non-Hispanic Blacks, those who were single, those with severe levels of baseline depressive symptoms, and those with severe levels of anxiety symptoms demonstrated greater amounts of decline in anxiety symptoms than Non-Hispanic Whites, those who were married/partnered/cohabiting, those with minimal levels of baseline depressive symptoms, and those with moderate levels of baseline anxiety symptoms. Participants identifying as sexual minorities, those with private or government-based health insurance, and those in concurrent mental health treatment demonstrated lower amounts of decline in anxiety symptoms than heterosexuals, those with no health insurance or who declined to answer the health insurance question, and those without concurrent mental health treatment, respectively. PPl analyses largely confirmed the full sample findings, with the exception of concurrent mental health treatment reaching marginal significance (p = .052, Supplementary Table 2).

After removing race/ethnicity, employment, and insurance status from consideration due to multicollinearity, the final multiple regression model indicated significant differences in the amount of decrease in anxiety symptoms at Week 8 by sexual orientation, marital status, baseline anxiety symptom severity category, and concurrent mental health treatment (Table 5) in the same direction as those previously described in each bivariate regression model.

Table 4: Unadjusted Bivariate Linear Regression Models of change scores from baseline to Week 8 in anxiety symptoms among those with clinically elevated levels of baseline anxiety symptoms: GAD-7 ≥ 10

	GAD-7: Week 8 Change Scores
Characteristics of Interest	Estimates	95% CI	p-value
Use of WB-LIFE in at least 4 of 8 weeks	2.2	-0.25, 4.7	0.077
Age	0.03	-0.07, 0.14	0.5
Race/Ethnicity Non-Hispanic White	Reference Level
Non-Hispanic Black	-5.7	-08.2, -3.2	<0.001
Other	-0.12	-3.0, 2.8	>0.9
Sex at Birth Male	Reference Level
Female	1.8	-1.0, 4.6	0.2
Sexual Orientation Heterosexual	Reference Level
Sexual Minority	3.7	0.65, 6.8	0.018
Education High School	Reference Level
College Degree	-1.7	-5.3, 1.9	0.3
Graduate or postgraduate degree	-1.8	-5.7, 2.1	0.4
Some college or technical school	-0.28	-4.5, 3.9	0.9
Employment Full Time	Reference Level
Not Employed	2.1	-1.0, 5.2	0.2
Other	1.2	-2.2, 4.7	0.5
Part Time Employed	2.3	-2.2, 6.8	0.3
Marital Status Married/Partnered/Cohabiting	Reference Level
Divorced/Separated/Widowed	-1.5	-5.8, 2.8	0.5
Single	-2.9	-5.4, -0.37	0.025
Health Insurance No Insurance/Prefer not to answer	Reference Level
Government based Insurance	5.1	2.1, 8.0	<0.001
Private Insurance	6.8	3.9, 9.6	<0.001
BL Depressive Symptom Severity Minimal	Reference Level
Mild	-1.4	-6.2, 3.4	0.6
Moderate	-0.71	-5.3, 3.9	0.8
Moderate-Severe	-0.12	-4.6, 4.4	>0.9
Severe	-7.3	-12, -2.7	0.002
BL Anxiety Symptom Severity Moderate	Reference Level
Severe	-4.9	-7.1, -2.7	<0.001
Concurrent Mental Health Treatment*	3.2	0.90, 5.6	0.007

BL = baseline; GAD-7 = Generalized Anxiety DIsorder-7 item scale

* Concurrent mental health treatment = any psychotherapy or psychotropic medication use at any time during the study

Table 5. Adjusted Linear Regression Model (Multiple Regression) of change scores from baseline to Week 8 in anxiety symptoms among those with clinically elevated levels of baseline anxiety symptoms: GAD-7 ≥ 10

	GAD-7: Week 8 Change Scores
Characteristics of Interest	Estimates	95% CI	p-value
(Intercept)	-4.68	-10.40 – 1.03	0.107
Used the app in at least 4 of 8 weeks	1.26	-1.06 – 3.59	0.282
Age	0	-0.11 – 0.11	0.975
Sex at Birth: Male	Reference Level
Female	-1.76	-4.63 – 1.10	0.224
Sexual Orientation: Heterosexual	Reference Level
Sexual Minority	4.47	1.20 – 7.74	0.008
Education Level: High School	Reference Level
College Degree	-1.7	-4.79 – 1.39	0.278
Graduate or postgraduate degree	-1.12	-4.72 – 2.48	0.536
Some college or technical school	-0.55	-4.38 – 3.28	0.776
Marital Status: Married/Partnered/Cohabiting	Reference Level
Divorced/Separated/Widowed	-2.85	-7.30 – 1.61	0.208
Single	-4.19	-6.69 – -1.68	0.001
BL Anxiety Symptom Severity: Moderate	Reference Level
Severe	-3.88	-6.14 – -1.62	0.001
Concurrent Mental Health Treatment*	3.2	0.95 – 5.45	0.006

BL = baseline; GAD-7 = Generalized Anxiety DIsorder-7 item scale

* Concurrent mental health treatment = any psychotherapy or psychotropic medication use at any time during the study

NOTE: Final models did not consider race/ethnicity, employment, and insurance status due to multicollinearity.

Safety

No study participant reported an AE or serious adverse event (SAE) during the study. Over the course of the study, 127,773 messages were sent by the 256 participants. A subset of 71 participants provided text inputs containing language suggestive of a potential crisis. Of these participants, four confirmed being in a crisis situation and were automatically provided with resources and support as described in the Safety section of the Methods.

In this single-arm exploratory trial of WB-LIFE, an intervention employing an NLP-supported relational agent, the objective was to explore the magnitude of the reduction in self-reported depressive (PHQ-8) and anxiety (GAD-7) symptoms between baseline and Week 8 EOI as well as the associative relationships between demographic and clinical characteristics and each outcome, among those that had elevated scores on these measures at baseline. On average, study participants with elevated depressive symptoms at baseline experienced significant declines in self-reported depressive symptoms across the intervention period; study participants with elevated anxiety symptoms at baseline experienced significant declines in self-reported anxiety symptoms across the intervention period. In addition, analyses revealed significant associations between depressive and anxiety symptom changes and demographic and clinical characteristics in both bivariate and, in many cases, multiple regression models. The high assessment completion and WB-LIFE utilization rates are notable given that attrition remains a perennial problem in mobile health app studies [45], with one recent systematic review of depression DMHI app RCTs suggesting that dropout rates may range from 26–48%, even when participant reimbursement was included in the analysis [46]. Participants in the present study were remunerated to complete assessments, but WB-LIFE’s ability to establish rapidly and maintain therapeutic alliance [22] may have contributed to this high participant app utilization rate and warrants further study.

In the absence of a control group, it is difficult to draw conclusions about the efficacy of WB-LIFE, especially given evidence that approximately one-third of untreated major depression or generalized anxiety disorder cases may spontaneously remit within 6 months [47–49]. However, in both the clinically elevated depressive and anxiety groups, we noted significant decreases in self-reported symptoms at Week 8. The mean level of change was greater than seven points on both change in PHQ-8 among those with clinically elevated depressive symptoms at baseline as well as change in GAD-7 among those with clinically elevated anxiety symptoms at baseline. This is noteworthy given that these outcomes were two points greater than the typically accepted change score cutoff of five points to define a clinically significant response on these measures [50].

Our adjusted analyses did not find significant associations between sex at birth, age, educational background, employment, and clinical outcomes, which is consistent with prior psychotherapeutic and DMHI meta-analyses and systematic reviews [8, 51–53]. However,

several studies of internet-based CBT (iCBT) have found better depression outcomes for females [26, 54] and greater probability of non-response in anxiety disorders and depression for males [55]. This could imply that DMHIs incorporating relational agents exhibit a more uniform efficacy profile than iCBT across demographic groups. Alternatively, our results may simply support the finding that age and sex at birth may be more important to the epidemiology of depression than to prognosis with an intervention [53].

The associations that were found in adjusted analyses between significant improvements in depression and anxiety symptoms and sexual orientation, concurrent mental health treatment, marital status, and baseline symptom severity suggest that these may be demographic predictors of intervention response to DMHIs employing relational agents. However, a series of purpose-designed follow-up studies to test each of these putative predictors of clinical outcome in a randomized clinical trial (RCT) design is needed before any definitive conclusions can be made. Given that LGBTQ individuals have demonstrated higher rates of depression and anxiety [56] that are further compounded by access barriers and stigma [31], DMHIs may present an unique opportunity to provide safe and accessible support to this group. LGBTQ individuals are also more likely to seek health information with digital resources than their non-LGBTQ counterparts [57] and a high proportion of people accessing digital text-based support services identify as belonging to sexual and gender minorities [58]. Thus, further studies among this group should be a research priority. Finally, our study noted significant differences in the association of concurrent care involvement with changes in anxiety and depression symptoms, but this concurrent care was received independently and therefore no details on its characteristics (e.g., description of intervention, frequency, fidelity to psychotherapeutic treatment model, modality of delivery, dose, and route of psychopharmacological agents) were available. Future studies should test if this association holds in a true blended care model that tightly integrates the relational-agent-delivered DMHI with human support.

Further, historical findings regarding the link between marital status and psychotherapy outcomes have been decidedly mixed [59], but in this study we found that single or divorced/separated/widowed individuals in the elevated depression group and single individuals in the elevated anxiety group experienced more improved outcomes compared to their married/partnered/cohabiting peers in adjusted analyses. These results are somewhat consistent with a study of an online cognitive behavior intervention compared to a waiting list control that demonstrated greater improvements in depression outcomes at 4 months among separated, widowed, or divorced individuals as compared to married participants [60]. Due to the importance of social support in mental health treatment [51], a more detailed assessment of factors such as relationship quality may better explain the relationship between marital status and clinical outcomes [51].

Finally, we found that greater baseline depression and anxiety symptom severity was associated with greater symptom improvements in the elevated depression and anxiety groups, respectively. It is notable that 50% of the elevated depression group reported either moderately severe or severe levels of baseline symptoms and 44% of the elevated anxiety groups reported severe symptoms. DMHIs have often been considered as interventions for mild to moderate symptoms [61], but these findings may suggest promise for populations with a broader range of severity. While it is true that those with elevated symptoms have more room to improve [8], we nonetheless believe that this finding suggests that there may be clinical utility for programs like this beyond mild to moderate severity levels with adequate safety checks in place.

Strengths and Limitations

The present study adds to the extant mental health outcomes literature by extending the investigation of important demographic and clinical characteristics in DMHI interventions to one delivered specifically via a relational agent. To our knowledge, this is the first study that has been expressly designed for this purpose. The recruitment of a demographically diverse naturalistic sample allowed a broad assessment of characteristics that have been traditionally underexplored in digital health studies, such as racial/ethnic membership, sexual orientation, employment status, and health insurance status. There has been increasing attention to the lack of diversity in marginalized and underserved participants involved in DMHI studies [31], and this study notably recruited much higher proportions of sexual minorities [62] and non-Hispanic Black [63] individuals than are found in the U.S. population. Despite a generally diverse sample, it should be noted that this sample was about 75% female sex at birth, a proportion that is not unusual for health behavior studies [64]. Greater recruitment based on gender identity and participation of males would improve sample diversity in future studies to allow for greater generalization of findings.

Several limitations must be noted. First, the study was not an RCT and did not include a control group, so there may have been other factors affecting the efficacy of the intervention. For example, researchers have identified user expectancies related to digital device use (a.k.a.“digital placebo effect”) that may affect mental health outcomes [65]. Subgroup differences observed in the present study are preliminary and should ideally be tested in RCTs. Second, because the intervention was delivered over an 8-week period, the longer-term durability of any effects is not known. Changes that did occur might be explained by measurement reactivity, as small effects have been observed in health behavior change [66] and digital health studies [67]. Symptom improvements may also be explained in part by spontaneous remission [47, 48, 68] or other participant characteristics that were not assessed as part of the study. Third, symptoms and concurrent treatments were assessed via self-report and there was no independent confirmation of participants' clinical presentation or therapeutic involvements. Although about half of the sample reported receiving concurrent treatment (consistent with national findings that only about half of those in need receive mental health care [69]), we do not know the nature, duration, or intensity of psychotherapies or psychotropic medications received. In addition, although we collected information about both psychotherapy and psychotropic medications separately, the sample size was not sufficient to make comparisons between these modalities and so they were pooled for analysis. Future studies would ideally collect detailed information about the psychotherapies being received, as well as full details about concurrent psychotropic medications. Fourth, although demographically diverse for a naturalistic sample, the study population was nonetheless majority college or advanced degree holders and employed. Additional studies targeting the mental health needs of those with lower levels of educational attainment and/or who may be facing under- or unemployment are needed. Finally, our definition of “per protocol” as using WB-LIFE in at least 4 of 8 study weeks and completing EOS depressive and anxiety symptom assessments may not have been granular enough to account fully for achieved outcomes or to reveal unique patterns of disengagement. Exploring alternative engagement metrics would enhance our understanding of relational agent usage and the relationship of outcomes to user characteristics.

The present study suggests early promise for a DMHI that employs an easily scalable NLP-based relational agent to deliver elements of CBT, DBT, and IPT to a diverse naturalistic sample experiencing self-reported symptoms of depression and/or anxiety. Although exploratory in nature, this study reveals intriguing areas for further research that can help optimize the unique role DMHIs that include relational agents can play in filling a mental health service gap that has only widened in the wake of the COVID-19 pandemic.

ACT: Acceptance and commitment therapy

AE/SAE: adverse and serious adverse events

CBT: Cognitive behavioral therapy

DASS-21: Depression Anxiety Stress Scale-21

DBT: Dialectical Behavior Therapy

DMHI: Digital Mental Health Intervention

EDC: Castor Electronic Data Capture

EOI: End of intervention

EOS: End of study

GAD-7: Generalized Anxiety Disorder

iCBT: Internet-based cognitive behavior therapy

IPT: Interpersonal Psychotherapy

ITT: intention to treat

LGBTQ: lesbian, gay, bisexual, transgender, queer or questioning

MBCT: mindfulness-based cognitive therapy

NLP: natural language processing

PHQ-8: Patient Health Questionnaire

PP: Per protocol

RCT: Randomized controlled trials

U.S.: United States

Ethics approval and consent to participate

This study was conducted in accordance with the principles of the Declaration of Helsinki. All study procedures were approved by the WIRB-Copernicus IRB Group (WCG) institutional review board [Protocol #20216751] on January 20, 2022. All participants provided informed consent.

Consent for publication

Not applicable.

Availability of data and materials

The datasets underlying the results of this study are confidential and include proprietary commercial information. Requests can be submitted to the corresponding author and will be subject to review and approval.

Competing interests

Authors EC, AW, TYM, SP, AR, AKQ, and VFH are all employees of Woebot Health. TYM is an employee of the US Department of Veterans Affairs. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States government.

Funding

This study was funded by Woebot Health.

Authors' contributions

EC, AW, TYM, SP, AR, AKQ, and VFH contributed to the conceptualization, study design, implementation, and manuscript preparation of this study. AW and VFH reviewed the study design and provided the statistical analysis plan. AW conducted the statistical analysis. All authors reviewed, edited, and approved the final manuscript.

Acknowledgements

We would like to thank Robbert Zusterzeel, M.D., Ph.D., MPH, Alison Darcy, Ph.D., and Megan Flom, Ph.D. for their helpful comments, as well as Adam Platt, M.Sc. for his assistance in preparing the manuscript and figures.

GBD 2019 Mental Disorders Collaborators. Global, regional, and national burden of 12 mental disorders in 204 countries and territories, 1990–2019: a systematic analysis for the Global Burden of Disease Study 2019. Lancet Psychiatry. 2022;9(2):137–50. https://doi.org/10.1016/S2215-0366(21)00395-3.
COVID-19 Mental Disorders Collaborators. Global prevalence and burden of depressive and anxiety disorders in 204 countries and territories in 2020 due to the COVID-19 pandemic. Lancet. 2021; 398(10312):1700-12. https://doi.org/10.1016/S0140-6736(21)02143-7.
Evans AC, Bufka LF. The critical need for a population health approach: addressing the nation’s behavioral health during the COVID-19 pandemic and beyond. Prev Chronic Dis. 2020;17:E79. http://dx.doi.org/10.5888/pcd17.200261.
World Health Organization. The impact of COVID-19 on mental, neurological and substance use services: results of rapid assessment. Geneva: World Health Organization; 2020. Available from: https://www.who.int/publications-detail-redirect/978924012455 [cited 14 Dec 2022].
Philippe TJ, Sikder N, Jackson A, Koblanski ME, Liow E, Pilarinos A, et al. Digital health interventions for delivery of mental health care: systematic and comprehensive meta-review. JMIR Ment Health. 2022;9(5):e35159. https://doi.org/10.2196/35159.
Sorkin DH, Janio EA, Eikey EV, Schneider M, Davis K, Schueller SM, et al. Rise in use of digital mental health tools and technologies in the United States during the COVID-19 pandemic: survey study. J Med Internet Res. 2021;23(4):e26994. https://doi.org/10.2196/26994.
Lattie EG, Stiles-Shields C, Graham AK. An overview of and recommendations for more accessible digital mental health services. Nat Rev Psychol. 2022;1:87–100. https://doi.org/10.1038/s44159-021-00003-1.
Moshe I, Terhorst Y, Philippi P, Domhardt M, Cuijpers P, Cristea I, et al. Digital interventions for the treatment of depression: a meta-analytic review. Psychol Bull. 2021;147(8):749–86. https://doi.org/10.1037/bul0000334.
Pauley D, Cuijpers P, Papola D, Miguel C, Karyotaki E. Two decades of digital interventions for anxiety disorders: a systematic review and meta-analysis of treatment effectiveness. Psychol Med. 2021;1–13. https://doi.org/10.1017/S0033291721001999.
Serrano-Ripoll MJ, Zamanillo-Campos R, Fiol-DeRoque MA, Castro A, Ricci-Cabello I. Impact of smartphone app–based psychological Iiterventions for reducing depressive symptoms in people with depression: systematic literature review and meta-analysis of randomized controlled trials. JMIR MHealth UHealth. 2022;10(1):e29621. https://doi.org/10.2196/29621.
Boucher EM, Harake NR, Ward HE, Stoeckl SE, Vargas J, Minkel J, et al. Artificially intelligent chatbots in digital mental health interventions: a review. Expert Rev Med Devices. 2021;18(sup1):37–49. https://doi.org/10.1080/17434440.2021.2013200.
Parmar P, Ryu J, Pandya S, Sedoc J, Agarwal S. Health-focused conversational agents in person-centered care: a review of apps. NPJ Digit Med. 2022;5(1):21. https://doi.org/10.1038/s41746-022-00560-6.
Vaidyam AN, Wisniewski H, Halamka JD, Kashavan MS, Torous JB. Chatbots and conversational agents in mental health: a review of the psychiatric landscape. Can J Psychiatr. 2019;64(7):456–64. https://doi.org/10.1177/0706743719828977.
Gaffney H, Mansell W, Tai S. Conversational agents in the treatment of mental health problems: mixed-method systematic review. JMIR Ment Health. 2019;6(10):e14166. https://doi.org/10.2196/14166.
Beatty C, Malik T, Meheli S, Sinha C. Evaluating the therapeutic alliance with a free-text CBT conversational agent (Wysa): a mixed-methods study. Front Digit Health. 2022;4:847991. https://doi.org/10.3389/fdgth.2022.847991.
Schroeder J, Wilkes C, Rowan K, Toledo A, Paradiso A, Czerwinski M, et al. Pocket skills: a conversational mobile web app to support dialectical behavioral therapy. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. New York, NY, USA: 2018. https://doi.org/10.1145/3173574.3173972
Fulmer R, Joerin A, Gentile B, Lakerink L, Rauws M. Using psychological artificial intelligence (Tess) to relieve symptoms of depression and anxiety: randomized controlled trial. JMIR Ment Health. 2018;5(4):e64. https://doi.org/10.2196/mental.9782.
Denecke K, Abd-Alrazaq A, Househ M. Artificial intelligence for chatbots in mental health: opportunities and challenges. In: Househ M, Borycki E, Kushniruk A, editors. Multiple Perspectives on Artificial Intelligence in Healthcare. Cham: Springer International Publishing; 2021. p. 115–28. https://doi.org/10.1007/978-3-030-67303-1_10.
Abd-Alrazaq AA, Rababeh A, Alajlani M, Bewick BM, Househ M. Effectiveness and safety of using chatbots to improve mental health: systematic review and meta-analysis. J Med Internet Res. 2020;22(7):e16021. https://doi.org/10.2196/16021.
Bickmore T, Gruber A. Relational agents in clinical psychiatry. Harv Rev Psychiatry. 2010;18(2):119–30. https://doi.org/10.3109/10673221003707538.
Chaudhry BM, Islam A. A mobile application-based relational agent as a health professional for COVID-19 patients: design, approach, and implications. Int J Environ Res Public Health. 2022;19(21):13794. https://doi.org/10.3390/ijerph192113794.
Darcy A, Daniels J, Salinger D, Wicks P, Robinson A. Evidence of human-level bonds established with a digital conversational agent: cross-sectional, retrospective observational study. JMIR Form Res. 2021;5(5):e27868. https://doi.org/10.2196/27868.
Wampold BE. How important are the common factors in psychotherapy? An update. World Psychiatry. 2015;14(3):270–7. https://doi.org/10.1002/wps.20238.
Hedman E, Andersson E, Lekander M, Ljótsson B. Predictors in internet-delivered cognitive behavior therapy and behavioral stress management for severe health anxiety. Behav Res Ther. 2015;64:49–55. https://doi.org/10.1016/j.brat.2014.11.009.
El Alaoui S, Ljótsson B, Hedman E, Kaldo V, Andersson E, Rück C, et al. Predictors of symptomatic change and adherence in internet-based cognitive behaviour therapy for social anxiety disorder in routine psychiatric care. PLoS ONE. 2015;10(4):e0124258. https://doi.org/10.1371/journal.pone.0124258.
Donker T, Batterham PJ, Warmerdam L, Bennett K, Bennett A, Cuijpers P, et al. Predictors and moderators of response to internet-delivered interpersonal psychotherapy and cognitive behavior therapy for depression. J Affect Disord. 2013;151(1):343–51. https://doi.org/10.1016/j.jad.2013.06.020.
Catarino A, Bateup S, Tablan V, Innes K, Freer S, Richards A, et al. Demographic and clinical predictors of response to internet-enabled cognitive–behavioural therapy for depression and anxiety. BJPsych Open. 2018;4(5):411–8. https://doi.org/10.1192/bjo.2018.57.
Edmonds M, Hadjistavropoulos HD, Schneider LH, Dear BF, Titov N. Who benefits most from therapist-assisted internet-delivered cognitive behaviour therapy in clinical practice? Predictors of symptom change and dropout. J Anxiety Disord. 2018;54:24–32. https://doi.org/10.1016/j.janxdis.2018.01.003.
Warmerdam L, Van Straten A, Twisk J, Cuijpers P. Predicting outcome of internet-based treatment for depressive symptoms. Psychother Res. 2013;23(5):559–67. https://doi.org/10.1080/10503307.2013.807377.
Anderson-Lewis C, Darville G, Mercado RE, Howell S, Di Maggio S. mHealth technology use and implications in historically underserved and minority populations in the United States: systematic literature review. JMIR MHealth UHealth. 2018;6(6):e128. https://doi.org/10.2196/mhealth.8383.
Schueller SM, Hunter JF, Figueroa C, Aguilera A. Use of digital mental health for marginalized and underserved populations. Curr Treat Options Psychiatry. 2019;6(3):243–55. https://doi.org/10.1007/s40501-019-00181-z.
Mohr DC, Kwasny MJ, Meyerhoff J, Graham AK, Lattie EG. The effect of depression and anxiety symptom severity on clinical outcomes and app use in digital mental health treatments: meta-regression of three trials. Behav Res Ther. 2021;147:103972. https://doi.org/10.1016/j.brat.2021.103972.
Fiol-DeRoque MA, Serrano-Ripoll MJ, Jiménez R, Zamanillo-Campos R, Yáñez-Juan AM, Bennasar-Veny M, et al. A mobile phone–based intervention to reduce mental health problems in health care workers during the COVID-19 pandemic (PsyCovidApp): randomized controlled trial. JMIR MHealth UHealth. 2021;9(5):e27039. https://doi.org/10.2196/27039.
Fitzpatrick KK, Darcy A, Vierhile M. Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial. JMIR Ment Health. 2017;4(2):e19. https://doi.org/10.2196/mental.7785.
Prochaska JJ, Vogel EA, Chieng A, Kendra M, Baiocchi M, Pajarito S, et al. A therapeutic relational agent for reducing problematic substance use (Woebot): development and usability study. J Med Internet Res. 2021;23(3):e24850. https://doi.org/10.2196/24850.
Nicol G, Wang R, Graham S, Dodd S, Garbutt J. Chatbot-delivered cognitive behavioral therapy in adolescents with depression and anxiety during the COVID-19 pandemic: feasibility and acceptability study. JMIR Form Res. 2022;6(11):e40242. https://doi.org/10.2196/40242.
Darcy A, Beaudette A, Chiauzzi E, Daniels J, Goodwin K, Mariano TY, et al. Anatomy of a Woebot® (WB001): agent guided CBT for women with postpartum depression. Expert Rev Med Devices. 2022;19(4):287–301. https://doi.org/10.1080/17434440.2022.2075726.
Suharwardy S, Ramachandran M, Leonard S, Gunaseelan A, Robinson A, Darcy A, et al. Effect of an automated conversational agent on postpartum mental health: a randomized, controlled trial. Poster presented at: Society for Maternal-Fetal Medicine's 40th Annual Pregnancy Meeting; 2020 Feb 3-8; Grapevine, TX.
Prochaska JJ, Vogel EA, Chieng A, Baiocchi M, Maglalang DD, Pajarito S, et al. A randomized controlled trial of a therapeutic relational agent for reducing substance misuse during the COVID-19 pandemic. Drug Alcohol Depend. 2021;227:108986. https://doi.org/10.1016/j.drugalcdep.2021.108986.
Castor EDC. Castor Electronic Data Capture. 2019. Available from: https://castoredc.com.
Kroenke K, Strine TW, Spitzer RL, Williams JBW, Berry JT, Mokdad AH. The PHQ-8 as a measure of current depression in the general population. J Affect Disord. 2009;114(1–3):163–73. https://doi.org/10.1016/j.jad.2008.06.026.
Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16(9):606–13. https://doi.org/10.1046/j.1525-1497.2001.016009606.x.
Spitzer RL, Kroenke K, Williams JBW, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092–7. https://doi.org/10.1001/archinte.166.10.1092.
R Core Team. R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2021. Available from: https://www.R-project.org/.
Amagai S, Pila S, Kaat AJ, Nowinski CJ, Gershon RC. Challenges in participant engagement and retention using mobile health apps: literature review. J Med Internet Res. 2022;24(4):e35120. https://doi.org/10.2196/35120.
Torous J, Lipschitz J, Ng M, Firth J. Dropout rates in clinical trials of smartphone apps for depressive symptoms: a systematic review and meta-analysis. J Affect Disord. 2020;263:413–9. https://doi.org/10.1016/j.jad.2019.11.167.
Whiteford HA, Harris MG, McKeon G, Baxter A, Pennell C, Barendregt JJ, et al. Estimating remission from untreated major depression: a systematic review and meta-analysis. Psychol Med. 2013;43(8):1569–85. https://doi.org/10.1017/S0033291712001717.
Wittchen HU, Zhao S, Kessler RC, Eaton WW. DSM-III-R generalized anxiety disorder in the National Comorbidity Survey. Arch Gen Psychiatry. 1994;51(5):355–64. https://doi.org/10.1001/archpsyc.1994.03950050015002.
Wittchen HU, Hoyer J. Generalized anxiety disorder: nature and course. J Clin Psychiatry. 2001;62(Suppl 11):15–9; discussion 20-1.
Kroenke K, Spitzer RL. The PHQ-9: a new depression diagnostic and severity measure. Psychiatr Ann. 2002;32(9):509–15. https://doi.org/10.3928/0048-5713-20020901-06.
Tanguay-Sela M, Rollins C, Perez T, Qiang V, Golden G, Tunteng JF, et al. A systematic meta-review of patient-level predictors of psychological therapy outcome in major depressive disorder. J Affect Disord. 2022;317:307–18. https://doi.org/10.1016/j.jad.2022.08.041.
Amati F, Banks C, Greenfield G, Green J. Predictors of outcomes for patients with common mental health disorders receiving psychological therapies in community settings: a systematic review. J Public Health (Oxf). 2018;40(3):e375–87. https://doi.org/10.1093/pubmed/fdx168.
Buckman JEJ, Saunders R, Stott J, Arundell LL, O’Driscoll C, Davies MR, et al. Role of age, gender and marital status in prognosis for adults with depression: an individual patient data meta-analysis. Epidemiol Psychiatr Sci. 2021;30:e42. https://doi.org/10.1017/S2045796021000342.
Spek V, Nyklícek I, Cuijpers P, Pop V. Predictors of outcome of group and internet-based cognitive behavior therapy. J Affect Disord. 2008;105(1–3):137–45. https://doi.org/10.1016/j.jad.2007.05.001.
Rozental A, Andersson G, Carlbring P. In the absence of effects: an individual patient data meta-analysis of non-response and its predictors in internet-based cognitive behavior therapy. Front Psychol. 2019;10:589. https://doi.org/10.3389/fpsyg.2019.00589.
Plöderl M, Tremblay P. Mental health of sexual minorities. A systematic review. Int Rev Psychiatry. 2015;27(5):367–85. https://doi.org/10.3109/09540261.2015.1083949.
Bowen D, Jabson J, Kamen C. mHealth: an avenue for promoting health among sexual and gender minority populations? Mhealth. 2016;2(9):36. https://doi.org/10.21037/mhealth.2016.09.01.
Pisani AR, Gould MS, Gallo C, Ertefaie A, Kelberman C, Harrington D, et al. Individuals who text crisis text line: key characteristics and opportunities for suicide prevention. Suicide Life Threat Behav. 2022;52(3):567–82. https://doi.org/10.1111/sltb.12872.
Chen CK, Nehrig N, Chou LJ, McGowan R, Guyton AF, Mustafiz F, et al. Patient extratherapeutic interpersonal problems and response to psychotherapy for depression. Am J Psychother. 2019;72(4):101–22. https://doi.org/10.1176/appi.psychotherapy.20190005.
Button KS, Wiles NJ, Lewis G, Peters TJ, Kessler D. Factors associated with differential response to online cognitive behavioural therapy. Soc Psychiatry Psychiatr Epidemiol. 2012;47(5):827–33. https://doi.org/10.1007/s00127-011-0389-1.
Torous J, Bucci S, Bell IH, Kessing LV, Faurholt‐Jepsen M, Whelan P, et al. The growing field of digital psychiatry: current evidence and the future of apps, social media, chatbots, and virtual reality. World Psychiatry. 2021;20(3):318–35. https://doi.org/10.1002/wps.20883.
Jones JM. LGBT Identification in U.S. ticks up to 7.1%. 2022. Available from: https://news.gallup.com/poll/389792/lgbt-identification-ticks-up.aspx [cited 14 Dec 2022].
United States Census Bureau. 2020 Census illuminates racial and ethnic composition of the country. 2021. Available from: https://www.census.gov/library/stories/2021/08/improved-race-ethnicity-measures-reveal-united-states-population-much-more-multiracial.html [cited 14 Dec 2022].
Ryan J, Lopian L, Le B, Edney S, Van Kessel G, Plotnikoff R, et al. It’s not raining men: a mixed-methods study investigating methods of improving male recruitment to health behaviour research. BMC Public Health. 2019;19:814. https://doi.org/10.1186/s12889-019-7087-4.
Torous J, Firth J. The digital placebo effect: mobile mental health meets clinical psychiatry. Lancet Psychiatry. 2016;3(2):100–2. https://doi.org/10.1016/S2215-0366(15)00565-9.
Rodrigues AM, O’Brien N, French DP, Glidewell L, Sniehotta FF. The question–behavior effect: genuine effect or spurious phenomenon? A systematic review of randomized controlled trials with meta-analyses. Health Psychol. 2015;34:61–78. https://doi.org/10.1037/hea0000104.
König LM, Allmeta A, Christlein N, Van Emmenis M, Sutton S. A systematic review and meta-analysis of studies of reactivity to digital in-the-moment measurement of health behaviour. Health Psychol Rev. 2022;16(4):551–75. https://doi.org/10.1080/17437199.2022.2047096.
Hengartner MP. Is there a genuine placebo effect in acute depression treatments? A reassessment of regression to the mean and spontaneous remission. BMJ Evid Based Med. 2020;25(2):46–8. https://doi.org/10.1136/bmjebm-2019-111161.
Substance Abuse and Mental Health Services Administration. Highlights for the 2020 National Survey on Drug Use and Health. 2021. Available from: https://www.samhsa.gov/data/sites/default/files/2021-10/2020_NSDUH_Highlights.pdf [cited 11 Jan 2023].

Competing interest reported. Authors EC, AW, TYM, SP, AR, AKQ, and VFH are all employees of Woebot Health. TYM is an employee of the US Department of Veterans Affairs. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States government.

Demographic and clinical characteristics associated with anxiety and depressive symptom outcomes in users of a digital mental health intervention incorporating a relational agent

Status:

Journal Publication

Version 1

Abstract

Figures

Background

Methods

Clinical Characteristics

Results

Discussion

Strengths and Limitations

Conclusions

Abbreviations

Declarations

References

Additional Declarations

Supplementary Files

Status:

Journal Publication

Version 1