Participants
Participants (N = 180) will be primary caregivers who are interested in seeking additional information about youth anxiety; specifically, caregivers who attend a presentation on youth anxiety at their youth’s school. Schools located in a metropolitan area in the northeastern United States will be recruited via their school mental health workers/other school administrators. School administrators will be contacted via email; local school partners (i.e., school psychologists and social workers) will assist with school recruitment as needed. To increase the racial, ethnic, and financial diversity of the sample, schools will only be contacted if they had at least 60% minority student enrollment or at least 60% of students eligible for school lunch [84]. School administrators will advertise presentations as they advertise other school events (e.g., email list, Facebook groups). To be eligible for this study, participants must be least 18 years of age, be fluent in English, be the primary caregiver of a youth aged 5 to 18 years, and have a child at one of the schools offering a presentation. Caregivers will be cluster randomized by school using restricted randomization with Excel’s random number generator. Randomization will occur after the school has agreed to participate in the study, but before caregivers enroll in the study. The principal investigator (clinical psychology candidate with a master’s degree) will randomize schools to the presentation condition and will enroll all participants. Neither the researchers nor the participants are blinded to study condition. Participants will be paid $20 to attend the presentation and complete the pre- and post-presentation questionnaires; $10 to complete the three-month follow-up questionnaire; and $20 for the qualitative interview.
Key Opinion Leaders
To select the KOLs, the principal investigator will contact the school parent-teacher association (or a similar parent group), and ask, “please nominate a caregiver who is well-known and respected within your community, and who reflects the diversity of the school as a whole.” If a school does not have an active parent teacher association (or similar group of active parents), the school staff may select the KOL. The KOLs do not necessarily have to be a member of the parent-teacher association or have experience (professional or personal) with mental health. Previous research supports KOL nomination by knowledgeable community members (e.g., caregivers in the parent teacher association) as a valid method for identifying trusted individuals in the community [74,85]. The principal investigator will ask if the first KOL on the list is interested in participating in the project. The KOL must be willing to endorse CBT with exposures. Should the KOL decline to participate, the parent-teacher association will be asked to nominate a second caregiver KOL. There will be one KOL per school; the total number of KOLs will depend on the number of schools needed to recruit 180 caregiver participants.
KOLs from at least two schools in the KOL condition will participate in a two-hour feedback meeting with the principal investigator, with the goal of leveraging the KOLs to be champions of CBT. During this meeting, the KOLs will discuss their experiences with youth anxiety, factors about their communities that may affect how anxiety symptoms present or are understood, and how caregivers in their community typically seek therapy. The KOLs will be sent a draft of the presentation to review prior to the meeting. During the meeting, the principal investigator will review the presentation materials and encourage the KOLs to discuss their reactions and provide feedback. The KOLs will consider which strategies they can endorse as being effective (e.g., remaining calm when their child becomes emotional). Motivational interviewing techniques will be used should KOLs be skeptical about the value of CBT [86]. The principal investigator will then modify the outreach presentation based on KOL feedback. Presentations will be modified separately for each school, so the KOLs who meet together do not need to come to consensus on presentation content. Following the group KOL feedback meeting, the principal investigator will meet with each KOL individually to (1) review/approve the modifications made; (2) answer remaining KOL questions about the content; (3) determine which sections the KOL is comfortable presenting, and which strategies they are willing to endorse; and (4) give the KOL the opportunity to practice. KOL meetings will take place via zoom. KOLs will be paid $40 per hour (5 hours=$200 per KOL).
The KOL training checklist will be used to ensure that the KOL training is delivered consistently (see Appendix A). The principal investigator will complete this checklist following the KOL training. She will mark whether the group training discussed KOL experiences with youth anxiety and reviewed the presentation materials, as well as whether the phone call reviewed modifications made to the presentation, allowed the KOL to ask questions, determine which parts of the presentation the KOL will present, which strategies the KOL will endorse, and allows the KOL the chance to practice.
Outreach Conditions
Caregivers in both conditions will be invited to an outreach presentation, which lasts 75 minutes with an additional 15 minutes for caregiver questions. Presentations will occur in the evening via Zoom, separate from parent-teacher association meetings. Each presentation will include information about identifying anxiety disorders, strategies for caregivers to help their youth with anxiety, CBT for youth anxiety, and strategies for finding a therapist who uses CBT with exposures. Exposure therapy will be emphasized given that exposure therapy is underutilized by therapists in the community despite being a core ingredient of CBT [87]. The text on the presentations is written at a 5.3 grade reading level. Presentations will incorporate stigma reduction strategies, such as education to dispel myths, and behavioral decision-making tools to elicit hope, empowerment, and motivation [41,88,89]. Presentation content is manualized and is presented using PowerPoint.
Researcher-Only Condition
Half the schools will be cluster randomized to receive a researcher-facilitated presentation, led by two clinical psychology graduate students. Content will be the same for all schools randomized to the researcher-only condition. This is an active control condition. Researcher facilitated outreach presentations are one strategy research groups use to disseminate information to the community [24,56,90].
KOL Condition
The other half of the schools will receive a KOL co-facilitated presentations with the principal investigator (a clinical psychology PhD candidate). The KOLs will be introduced as a member from their school who has worked with the principal investigator to tailor the presentation to their community. Although the presentation is manualized and will contain the same core principles, content may vary by school in terms of specific examples and content emphasized based on KOL feedback. KOL will be encouraged to share personal stories and examples of how the presentation material can apply to the school community (to increase a sense of homophily to the KOL, as well as local relevance of the information).
Fidelity and Manipulation Checks
A 20-item Knowledge Test will assess caregivers’ knowledge of the content reviewed in the presentation (i.e., identifying anxiety disorders, strategies for caregivers to manage youth anxiety, EBPs to treat youth anxiety, and strategies for finding a therapist). The knowledge test is modeled after one to assess therapist training of CBT for anxiety [91]. Questions are true/false and multiple-choice format. Responses will be coded such that 1 = correct and 0 = incorrect, for a maximum of 20 points. The Knowledge Test will be used as a manipulation check to test participants’ understanding of the presentation material.
A content checklist will assess the core components of the presentation (i.e., identifying anxiety disorders, strategies for caregivers to help their youth with anxiety, how anxiety is treated, and strategies for finding a CBT therapist). A research assistant will function as an independent evaluator to complete this measure and evaluate the content of the outreach presentations. The research assistant will code for presenter and audience member self-disclosure about experience receiving therapy for themselves or their child. The research assistant also will record the total amount of time each presenter speaks. Two research assistants will be present for at least 20% of presentations; interrater reliability of the evaluators (κ) will be calculated.
Quantitative Measures
All questionnaires will be completed and stored on REDCap (a HIPAA secure platform [92]) hosted at Temple University. Participants will provide informed consent via REDCap before completing questionnaires. Figure 2 provides a summary of the schedule of enrolment, interventions, and assessments using the SPIRIT flow diagram [93]. All measures that were created for Project CHAT are in Appendix A and are elaborated below.
Treatment Seeking Evaluation
Pre and post presentation, caregivers will rate how likely they are to both seek a therapist for their child, as well as a therapist who uses exposure therapy, in the next three months. Rating scale ranges from 1 (very unlikely) to 5 (very likely). At the three-month follow-up assessment, parents will be asked if they have sought therapy for their youth since the presentation. If so, they will be asked if the child has started therapy, if they requested a therapist who uses exposure therapy, and for the name of their child’s therapist.
Knowledge about Seeking CBT
The Parent Engagement in Evidence-Based Services Questionnaire [94] is a 39-item measure of factors associated with seeking mental health care based on the theory of planned behavior [78]. Caregivers rate each statement on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree); some items are reverse coded. Caregiver ratings are summed to create five subscales [95]; this study will use the knowledge subscale to measure caregivers’ perceived understanding of how to seek EBPs (i.e., perceived behavioral control). On this subscale, higher scores indicate higher levels of perceived knowledge about seeking evidence-based practice. Evidence supports knowledge subscale’s internal consistency (α = .72) and convergent validity (r = .25-.41) [95].
Internalized Stigma
The Parents’ Internalized Stigma of Mental Illness Scale [96] is a 10-item measure of caregiver perception of internalized stigma for having a youth with a mental illness. Caregivers rate each statement on a 4-point Likert scale ranging from 1 (strongly disagree) to 4 (strongly agree); some items are reverse coded. Higher scores indicate higher levels of family stigma. The Parents’ Internalized Stigma of Mental Illness Scale has acceptable internal consistency (α = .76). It is an adaptation of the well-validated Internalized Stigma of Mental Illness Scale [97,98], which has demonstrated sensitivity to change in the expected direction after stigma reduction interventions [97].
Caregiver Attitudes about Cognitive Behavioral Therapy
The Caregiver Attitudes about CBT includes 18 strategies used in CBT for youth anxiety. Caregivers rate how helpful they believe each strategy would be for treating their child on a five-point scale ranging from 1 (very unhelpful) to 5 (very helpful). All items will be summed; higher scores indicate more favorable attitudes. Items were generated using an expert consensus (three clinical psychologists specializing in exposure treatment and one advanced doctoral candidate in clinical psychology). Some items were modeled on the Knowledge of Evidence Based Services Questionnaire [99].
Therapy Subjective Norms
The Therapy Subjective Norms Questionnaire is a six-item measure of caregiver perception of subjective norms for seeking therapy. It was modeled from previously used measures of subjective norms [100,101]. Caregivers rate each item on seven-point scale ranging from 1 (strongly disagree) to 7 (strongly agree). Three items assess injunctive norms (i.e., how other people would view an action the participant does; injunctive norms subscale), and three items assess descriptive norms (i.e., the participant’s view about what other people are doing; descriptive norms subscale). Items will be summed to create a score for overall subjective norms (all six items), as well as the injunctive and descriptive norms subscales; higher scores indicate more positive subjective norms about seeking therapy. Participants will complete two versions of this measure (12 items total): in one version, they will rate subjective norms related to seeking therapy, and in the other version, they will rate subjective norms related to seeking CBT.
Impression of Presenters
On the Relatability Evaluation, caregivers will rate each presenter (the KOL and the researcher, or the two researchers) on the following 10 dimensions: relatability, likeability, similarity, similarity in thinking, similarity of beliefs, credibility, trustworthiness, understanding of the local community, familiarity, and friendship. Scores will be given on a scale ranging from 1 (strongly disagree) to 5 (strongly agree). These items are based on characteristics of homophily [102].
Barriers to Seeking Treatment
The Barriers to Seeking Treatment questionnaire asks participants to indicate whether they agree with 21 potential barriers to treatment (yes/no). This questionnaire is adapted from the Collaborative Psychiatric Epidemiology Studies [103], and includes items relating to attitudinal barriers (e.g., wanted to handle on their own, stigma) and structural barriers (e.g., cost, transportation) [29]. Items were adapted to describe potential barriers caregivers may face (e.g., rather than saying “I need therapy”, the questionnaire was modified to read “my child needs therapy”).
Youth Anxiety
The Brief Revised Child Anxiety and Depression Scale-Parent Version is a 25-item caregiver report measure of anxiety and depressive symptoms [104]. Items are rated on a 4-point Likert-scale from 0 (never) to 3 (always). It yields three scores: Total Anxiety, Total Depression, and Total Anxiety and Depression. This study will use the Total Anxiety score. Previous research supports the Brief Revised Child Anxiety and Depression Scale total anxiety subscale’s internal reliability (α = .80 - .86), retest reliability (r = .85), convergent validity (r = .59), and discriminant validity for anxiety diagnoses (AUC = .81) [104].
Client Satisfaction
Caregivers will evaluate their satisfaction with the presentation using the Client Satisfaction Questionnaire [105]. This scale includes eight Likert-scale questions and three short answer questions. On the Likert-scale questions, caregivers will rate their level of satisfaction on a 4-point scale ranging from 1 to 4, with higher composite scores indicating greater program satisfaction. Psychometric analyses indicate excellent internal consistency (α = .93) and convergent validity (r = -.40 - .23) [105].
Demographics and Mental Health History
A demographics questionnaire will assess caregiver and youth age, gender, race, ethnicity, and country of origin; caregiver level of education, income, and religious service attendance; and youth health insurance status. The presenters also will indicate their age, gender, country of origin, number of children, and level of education to assess their similarity to participants. On the mental health history questionnaire, participants will indicate whether they or their youth have ever been diagnosed with or treated for a mental disorder, whether they or their youth have received CBT with exposures, and their level of satisfaction with their youth’s previous treatment experience.
Qualitative Interviews
After participants have completed the three-month follow-up questionnaire, 40 participants will be contacted to complete a qualitative interview via a Zoom videoconference. Participants will be purposefully sampled such that 20 participants who have sought treatment (10 per condition) and 20 who have not sought treatment (10 per condition) will be selected using Excel’s random number generator. Additional participants will be recruited until thematic saturation is reached [106].
Semi-structured interviews (see Appendix A for the interview guide) will be conducted by undergraduate research assistants (N = 4) using a funnel-approach, with open ended questions followed by specific required and optional probes for details [107]. Interviews will elicit information about barriers to seeking treatment, and the role of the presentation in reducing those barriers. Primary topics will include: (1) their perception of presenters; (2) ways in which the presenters affected their decision to seek treatment; (3) factors they considered when seeking treatment; (4) strategies they have used from the presentation; (5) their perception of exposure therapy; and (6) general ways that the mental health system could be improved to improve access to therapy. Interviews will close with a question asking for general additional feedback. Interviews will last approximately 30 minutes and will be digitally recorded via Zoom.
After each interview, the interviewer will rate the participant’s level of interest and involvement in answering the questions (1 = very low to 5 = very high), their understanding of the interview (1 = limited to 5 = complete), and their impression of the participant’s knowledge of the topics discussed (1 = highly questionable to 5 = highly knowledgeable). The interviewer also will comment on discrepancies in the interview and circumstances that may have affected quality of responses. Zoom transcripts of the interviews will be used, and a research assistant will check the transcription for accuracy. Transcripts will be deidentified.
Analytic Plan
Missing data
The primary analytic tool will be multilevel modeling using maximum likelihood estimation, which provides unbiased parameter estimates when data are missing at random. The missing at random assumption will be tested by multiple logistic regression analyses examining whether key predictors at baseline (i.e., Knowledge Test, Parent Engagement in Evidence-Based Services Questionnaire–knowledge subscale, Parents’ Internalized Stigma of Mental Illness Scale, Therapy Subjective Norms, Caregiver Attitudes about CBT, Treatment Seeking Evaluation, and demographics) are associated with study retention. Should analyses reveal that dropout is differentially associated with outcomes, multiple imputation will be used [108–110]. Every effort will be made to prevent missing data, such as by using REDCap options that remind participants to answer blank questions, and by emailing participants who have not completed all questionnaires.
Power Analysis
For Primary Aim 1, a Monte Carlo-based power estimate was derived using Mplus with 10,000 replications. For the sample size of 180, assuming a Type I error rate of 5%, a two-tailed test, statistical power was .83 to detect a medium-sized effect (r = .30) of randomization group on longitudinal changes, given an expectation of a small (r = .15) effect for the control group. For Primary Aim 2, power was calculated using G*Power. Given the brevity of the three-month follow-up questionnaire, a 10% attrition rate was assumed. Assuming a Type I error rate of 5%, a two-tailed test, and a 25% rate of seeking CBT in the researcher-only condition, statistical power was .82 to detect a medium effect (odd ratio = 1.72).
Data analysis and interpretation
Quantitative Analyses. Quantitative analyses will use multilevel modeling to account for the nesting of repeated measures within caregivers. Preliminary analyses will examine the effect of clustering of caregivers within schools. If schools account for more than 10% of variance in the outcomes after controlling for condition, a three-level multilevel model will be used to account for nesting of repeated measures within caregivers within schools.
Analyses will consider intention to seek CBT with exposures (Treatment Seeking Evaluation - Intention to seek CBT), subjective norms about seeking CBT (Therapy Subjective Norms Questionnaire–CBT), attitudes about CBT (Caregiver Attitudes about CBT), caregiver stigma about mental illness (Parents’ Internalized Stigma of Mental Illness Scale), and knowledge about how to seek EBPs for youth anxiety (Parent Engagement in Evidence-Based Services Questionnaire–Knowledge Subscale) as person-level dependent factors; condition (caregiver or researcher co-facilitator) as a person-level predictor; and time (pre- and post-presentation) as an observation-level predictor. In separate multilevel models, (a) intention to seek CBT, (b) Therapy Subjective Norms Questionnaire–CBT, (c) Caregiver Attitudes about CBT, (d) Parents’ Internalized Stigma of Mental Illness Scale, and (e) Parent Engagement in Evidence-Based Services Questionnaire–Knowledge subscale will be regressed on time, condition, and the interaction between time and condition; a random intercept will be included in all five multilevel models. A binary logistic regression will be conducted with CBT service seeking at the three-month follow-up (Treatment Seeking Evaluation - Actual CBT seeking) entered as the dependent variable, condition entered as the independent variable, and youth anxiety (Brief Revised Child Anxiety and Depression Scale–Total Anxiety) entered as a control variable. T-tests will be used to compare difference between conditions for each item on the Relatability Evaluation of the principal investigator. This study will examine caregiver demographic factors, youth anxiety (Brief Revised Child Anxiety and Depression Scale–Total Anxiety), racial similarity to the presenter (Demographics, same race), and self-disclosure (Content Checklist, self-disclosure), as potential moderators of the effect of presentation condition on intention to seek CBT. In separate multilevel models, intention to seek CBT will be regressed on time, condition, each potential moderators, and their three-way interaction.
Qualitative Analyses. The transcribed qualitative interviews will be entered into NVivo software for analysis. Qualitative analyses will use a direct content analysis approach [111]. The coding team will create an initial codebook using the primary topics asked in the qualitative interviews. Additional codes will be added to code text that does not fit into the initial categories, either to split the initial codes into two, or to create new codes. Coding will occur through a consensus process in which each transcript will be coded independently by two raters, who will arrive at consensus through discussion [112]. Thematic responses will be examined by both condition and by whether the caregiver has sought treatment for their youth (4 groups total).
Integration Procedures. Mixed methods integration will follow a QUAN à qual structure with an expansion approach [113]; quantitative methods being used to test hypotheses about the intervention and qualitative methods being used to contextual the results.