Participants
Participants were 847 junior college, undergraduate, and graduate students enrolled in a college or university in Kyoto or Nagoya, Japan, and participated in the Healthy Campus Trial (HCT) between September 2018 and May 2020. The HCT is a fully factorial randomized controlled trial to optimize smartphone cognitive behavioral therapy developed for mental health promotion and depression prevention in healthy college students. Details of the clinical trial are described in the protocol paper[9].
Measurement
Cognitive and behavioral skills
We used the following established questionnaires to integrate the five constructs of cognitive or behavioral skills. We have obtained written permission from the developers of all the original scales to use parts of their items in the CBT Skills Scale.
Self-monitoring (SM): From the Cognitive Behavioral Self-Monitoring 17-item scale originally developed by Tsuchida et al[14], we used the Cognitive Monitoring subscale for measuring SM skills. This scale contains questions such as, “I don't bother to think about how my actions relate to my feelings” and “Sometimes I don't understand how my actions and feelings are related to each other.” Each item was rated on a four-point Likert scale from 0 = very untrue of me to 3 = very true of me, with a total score between 0 and 15. Cronbach's α reported in a previous study of Japanese university students was 0.77, indicating sufficient internal consistency [14].
Cognitive restructuring (CR): The six highest loading items from the Competencies of the Cognitive Therapy Scale developed by Strunk et al[16] were used for measuring CR skills. The questions include, “When I became distressed because of a negative thought or feeling, I come up with a specific plan of action for what I could do to deal with it,” and “When something upset me, I paid attention to what I was thinking so I could have a more balanced view.” Items are rated from 0 = not very true about me, to 3 = very true about me, with a total score between 0 and 18. We confirmed the semantic equivalence of the Japanese and English versions by translation and back translation.
Behavioral activation (BA): We used the five-item Behavioral Activation Subscale of Behavioral Activation for Depression Scale – Short Form (BADS-SF) developed by Manos et al[15] for measuring BA skills. BADS-SF was translated to Japanese from the English version and validated in the Japanese population[19]. The scale includes items such as, “I'm satisfied with the amount and type of things I've done.” Items are rated from 0 = not very true in my mind to 3 = very true in my mind. Cronbach's α reported in a previous study of Japanese university students was 0.71, indicating sufficient internal consistency[19].
Assertiveness training (AT): We used the seven-item Self-Assertion subscale of the Adult Social Skills Scale developed by Aikawa et al[17] for measuring AT skills. The scale includes items such as, “I complain clearly when I’m made uncomfortable.” Items are rated between 0 = not very true about oneself to 3 = very true about oneself. Cronbach's α reported in a previous study of Japanese university students was 0.73, indicating sufficient internal consistency[17].
Problem-solving (PS): The six highest loading items of the Approach Avoidance Style subscale of the Problem-Solving Inventory[18] were used for measuring PS skills. A Japanese version of the inventory was translated and back translated. The scale includes items such as, “When making a decision, I weigh the consequences of each alternative and compare them against each other.” Items are rated from 0 = very untrue of me to 3 = very true of me, with a total score ranging from 9 to 18.
Mood and personality
Depression: The Patient Health Questionnaire-9 (PHQ-9)[20] was used for measuring depression. The PHQ-9 uses 9 items of diagnostic criteria for major depressive episode (MDE) in the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV), and is rated from 0 = not at all to 3 = nearly every day. The Japanese version of the PHQ-9 has been shown to be valid and reliable in primary care patients[21] and the university students[22].
Anxiety: The Generalized Anxiety Disorder-7 (GAD-7)[23] was used for measuring anxiety. The GAD-7 is an established measure including 7 items of anxiety, worry, and hypersensitiveness. The scale is rated from 0 = not at all to 3 = nearly every day. The Japanese version of the PHQ-9 has been shown to be valid and reliable in primary care patients with anxiety disorder or major depressive disorder[24].
Personality: For investigating the correlation between the CBT skills and personality traits, we used the short form of the Big Five Scale of Personality Traits (Big Five Scale), which is well established in reliability and validity, and is frequently used in Japan[25][26]. The Big Five Scale is rated from 0 = untrue for me to 4 = true for me on each of the five personality trait factors: Neuroticism, Extraversion, Openness, Conscientiousness, and Agreeableness.
Data collection
Data were collected as an assessment before participants were enrolled in the HCT and assigned to each intervention group, and all participants completed all questionnaires on the trial registration website and the smartphone CBT app on their smartphones. Of these, those participants who gave consent for the use of the data were used for the current analysis. The HCT is being conducted with the approval of the Ethics Committee of Kyoto University School of Medicine. We analyzed the cross-sectional data of baseline instruments of the trial in this investigation.
Statistical analysis
Data analysis was conducted with SPSS 25 (IBM Corp., Armonk, NY, USA) for investigating construct validity and internal consistency. Amos 22 (IBM SPSS Statistics, Chicago, IL, USA) was used for confirmatory factor analysis.
Structural validity
We first conducted an exploratory factor analysis (EFA) using maximum likelihood and Promax rotation to identify the factor structure of the five CBT skill scales collectively. Next, a confirmatory factor analysis (CFA) was performed based on the factor loadings of each factor to confirm the goodness-of-fit for the factor structure. Chi-squared (CMIN), goodness-of-fit index (GFI), adjusted goodness-of-fit index (AGFI), comparative fit index (CFI), and root mean square error of approximation (RMSEA) were calculated. The criterion for a good model fit was set to CMIN/df ≤ 2, GFI ≥ 0.95, AGFI ≥ 0.90, CFI ≥ 0.97, and RMSEA ≤ 0.05. An acceptable model fit was set to CMIN/df ≤ 3, GFI ≥ 0.90, AGFI ≥ 0.85, CFI ≥ 0.95, and RMSEA ≤ 0.08[27].
Reliability
Cronbach's α coefficients were calculated to test the internal consistency reliability of each of the five CBT skills scales. The α value considered to be of sufficient internal consistency ranges from 0.7 to 0.8.[28]
Construct validity
Pearson's correlation coefficients were calculated to analyze the correlation between the five CBT skill scales with the Big Five Scale, PHQ9, and GAD7 with symptoms of depression, anxiety, and specific personality traits.