Descriptive statistics
Responses distribution on five categories for each item in the K6 is presented in Table 1. We can see the symptoms distributed as a positive skewness. The majority of people have no symptoms, while only a few have severe symptoms. According to the cut point of 12/13, the prevalence of psychological distress among the current sample is 5.3%.
Table 1 Responses distribution on five categories
Item
|
0
|
1
|
2
|
3
|
4
|
1.Depressed
|
3521(54.59%)
|
2052(31.81%)
|
275(4.26%)
|
431(6.68%)
|
171(2.65%)
|
2.Nervous
|
4099(63.55%)
|
1782(27.63%)
|
193(2.99%)
|
296(4.59%)
|
80(1.24%)
|
3.Restless or fidgety
|
4007(62.12%)
|
1741(26.99%)
|
260(4.03%)
|
328(5.09%)
|
114(1.77%)
|
4.Hopeless
|
4650(72.09%)
|
1207(18.71%)
|
202(3.13%)
|
292(4.53%)
|
99(1.53%)
|
5.Everything was an effort
|
3813(59.12%)
|
1599(24.79%)
|
278(4.31%)
|
545(8.45%)
|
215(3.33%)
|
6.Worthless
|
4790(74.26%)
|
1142(17.71%)
|
164(2.54%)
|
251(3.89%)
|
103(1.60%)
|
Note. N=6450. 0= None of the time, 1=A little of the time, 2=Some of the time, 3=Most of the time, 4=All of the time
Examining factor structure
Assessment of dimensionality
The scalability of the K6 is presented in Table 2. For inter-item pairs, the inter-item scalability coefficients (Hij) range from 0.47 to 0.68. For items, the item scalability coefficients (Hi) ranged from 0.57 to 0.59. For the whole K6 scale, the scalability coefficient was 0.58 (SE=0.009). All the scalability coefficients were significantly greater than the conventional lower-bound value of 0.3. The results suggested the K6 should be considered as a scale of strong strength. The internal consistency of the six items was also excellent (Cronbach's alpha =0.87).
Table 2 Descriptive statistics of the items (upper panel) and the scale (lower panel) for the K6
Item
|
M
|
SD
|
Hj
|
SE
|
citc
|
1
|
0.71
|
1.01
|
0.581
|
0.011
|
0.72
|
2
|
0.52
|
0.86
|
0.555
|
0.011
|
0.70
|
3
|
0.57
|
0.92
|
0.590
|
0.010
|
0.76
|
4
|
0.45
|
0.88
|
0.584
|
0.011
|
0.74
|
5
|
0.72
|
1.09
|
0.574
|
0.010
|
0.71
|
6
|
0.41
|
0.85
|
0.592
|
0.011
|
0.74
|
M
|
|
|
3.38
|
|
|
SD
|
|
|
4.38
|
|
|
H
|
|
|
579
|
0.009
|
|
α
|
|
|
0.87
|
|
|
λ2
|
|
|
0.87
|
|
|
MS
|
|
|
0.87
|
|
|
LCRC
|
|
|
0.87
|
|
|
Note. N=6450. Hj=item-scalability coefficient; SE=standard error of item scalability coefficient; citc= corrected item-test correlation; H=total-scalability coefficient; α=Cronbach’s alpha; λ2=Guttman’s lambda-2; MS=Molenaar–Sijtsma method; LCRC=Latent Class Reliability Coefficient.
We further explored the dimensionality for all the six items by conducting iterative automated item selection procedure (AISP). The results were presented in Table 3. We followed the recommendation of Hemker et al. (1995), and set an initial value of lower bound c from 0 to 0.75 with increment steps of 0.05. For 0 ≤ c ≤ 0.55, all six items were selected to form one scale. For c=0.6, two scales emerged, including items 1-3 and items 4-6, respectively. For c=0.65, items 1 and 3 were unscalable. For c>0.7, all items were unscalable. The c value is suggested to set at 0.3 in practice, because the solution produced by the AISP is often hard to interpret when c ≥0.35 [30]. Therefore, the results from the AISP confirmed the unidimensionality of the K6.
Table 3 The results of automated item selection procedure for the K6
|
|
Item numbers
|
c
|
Results
|
Scale 1
|
Scale 2
|
Unscalable
|
0-0.55
|
1: 6
|
1-6
|
|
|
0.6
|
2:3,3
|
1-3
|
4-6
|
|
0.65
|
2:2, 2
|
2, 3
|
4, 6
|
1,5
|
0.7-0.75
|
0
|
|
|
1-6
|
Assessment of local independence and monotonicity
Moreover, we examined local independence and monotonicity to ensure the data were adequately fit to the Mokken scale. For local independence, no item-pair was flagged as locally dependent according to two indices (W1 and W2) calculated in the conditional association procedure [30]. That is, there is no evidence of violating local independence. For monotonicity, the results showed that no item violated the monotonicity assumption. Graphical analysis indicated that all items showed monotonical increases (see Figure 1).
Table 4 Output of assessment of monotonicity
Item
|
#ac
|
#vi
|
#zsig
|
crit
|
1
|
40
|
0
|
0
|
0
|
2
|
40
|
0
|
0
|
0
|
3
|
40
|
0
|
0
|
0
|
4
|
40
|
0
|
0
|
0
|
5
|
37
|
0
|
0
|
0
|
6
|
40
|
0
|
0
|
0
|
Note. N=6450. #ac = number of active pairs that were investigated; #vi = number of violations in which the item is involved; # zsig = number of significant z-values; crit = Crit value
Assessment of invariant item ordering
Graphically comparisons indicated that serval IRFs were almost identical, and it was hard to establish an invariant item ordering from visual inspection. A more rigorous method, increasing in transposition [40], was employed to investigate invariant item ordering, and the results suggested that Item 5 and Item 1 showed signs of violating invariant item ordering. Coefficient HT=0.09, is much less than the conventional criteria 0.3, which means that the item ordering is too inaccurate to be useful. Therefore, the invariant item ordering assumption is not supported. In sum, the Double Monotonicity model didn't fit the data well, while the three assumptions (unidimensionality, local independence, and monotonicity) of Monotone Homogeneity model were still met. People can be ordered on the latent trait according to their total score on the scale.
Reliability
Table 2 also provides reliability-estimates: coefficients of α =0.87, λ2=0.87, MS=0.87, and LCRC=0.87. All estimates are close to .9, and thus satisfactory. The corrected item-test correlations were satisfactory for all items, ranged from 0.64 to 0.70.
Sex differences
We also conducted the same analyses on the data from the male and the female subgroups separately. A similar pattern emerged for the scalability assessment among these two samples. Therefore, the K6 assesses psychological distress in a similar way and with a similar strength both sex.
Examining measurement invariance
Figure 2 illustrates the trait distributions of the male and the female. The male has lower mean scores than the female, but there is still a broad overlap. Table 5 presents the main results of DIF analysis. According to the LR χ2 test, Item 4 and Item 5 were marked for uniform DIF, but none was flagged for non-uniform DIF. Further examination of these two items revealed that for the same latent trait score, females are always rated with higher frequencies than males. For both items, the lower-left graph shows the uniform DIF was mainly caused by the fifth category threshold value (3.31vs.2.9, 2.57 vs 2.45). However, McFadden's pseudo R2 statistics (no more than 0.0011) indicated that the magnitude of DIF was very small for each item. Figure 3 is a graphical representation of the impact of all items and DIF items on the whole scale. The left one shows the impact of all six items, indicating a negligible difference across sex. The right one shows curves for the 2 DIF items, indicating that female score a bit higher when sex group-specific parameter estimates were used.
Table 5 Differential Item Functioning in the male and the female subgroups
Item
|
Uniform DIF
|
Non-uniform DIF
|
c122
|
ΔR2
|
Δβ12
|
c232
|
ΔR2
|
1
|
0.1523
|
0.0001
|
0.0024
|
0.5028
|
0.0000
|
2
|
0.0535
|
0.0003
|
0.0028
|
0.0406
|
0.0003
|
3
|
0.1448
|
0.0002
|
0.0018
|
0.0431
|
0.0003
|
4
|
0.0004
|
0.0011
|
0.0077
|
0.0240
|
0.0005
|
5
|
0.0056
|
0.0005
|
0.0062
|
0.4029
|
0.0000
|
6
|
0.9285
|
0.0000
|
0.0001
|
0.4288
|
0.0001
|
Additional exploratory factor analysis and confirmatory factor analysis
EFA yielded only one component that had an eigenvalue greater than one (eigenvalue λ=4.41), which explained 73.5% of the total variance. Both parallel analysis and scree plot also suggested the one-factor solution. All items had factor loadings greater than 0.8 (See Table 6). However, in terms of the goodness of model fit indices, the one-factor model didn't fit the data well, χ2=1336.664, df =9, CFI=0.972, TLI=0.953, and RMSEA=0.151 (90% CI 0.144, 0.158). A two-factor model ("Depressed", "Nervous", and "Restless or fidgety" on the first factor, while "Hopeless", "Everything was an effort ", and "Worthless "on the second factor) had a better and an acceptable fit to the data, χ2=18.912, df=4, CFI=1.00, TLI=0.999, and RMSEA=0.024 (90% CI 0.014, 0.035).
Table 6 Factor loadings of the K6 resulted from EFA and CFA
Item
|
EFA
|
CFA
|
|
One-factor model
|
Two-factor model
|
One-factor model
|
Two-factor model
|
|
Factor
|
Factor 1
|
Factor 2
|
Factor
|
Factor 1
|
Factor 2
|
1.Depressed
|
0.811
|
0.585
|
|
0.811
|
0.838
|
|
2.Nervous
|
0.828
|
0.907
|
|
0.828
|
0.839
|
|
3.Restless or fidgety
|
0.859
|
0.644
|
|
0.859
|
0.891
|
|
4. Hopeless
|
0.855
|
|
0.880
|
0.855
|
|
0.884
|
5. Everything was an effort
|
0.802
|
|
0.671
|
0.802
|
|
0.839
|
6. Worthless
|
0.857
|
|
0.931
|
0.857
|
|
0.882
|
In CFA, We tested the one-factor model and the two-factor model derived from EFA, as well as three other two-factor models proposed by Kessler et al. [6], Lee et al. [22] and Bessaha [7]. In Kessler et al's model, an item ("Everything was an effort") loads on the second factor, while all other five items load on the first factor. In Lee et al.’s model, three items (Nervous", "Restless or fidgety", and "Everything was an effort") load on the anxiety factor, while the rest three items ("Hopeless", "Depressed", and "Worthless") load on the depression factor. In Bessaha’s model, two items ("Nervous" and "Restless or fidgety") loaded on the anxiety factor, while all the other four items on the depression factor. The model goodness-of-fit indices are displayed in Table 7. Because Kessler et al’s model is not identified in CFA, the estimation is not reliable and not listed in the table. The fit indices suggested that the two-factor model derived from EFA in the present study is the only acceptable model. Both EFA and CFA showed that the two-factor model is the best.
Table 7 Model goodness-of-fit indices
Model
|
χ2
|
df
|
CFI
|
TLI
|
RMSEA
|
RMSEA 90%CI
|
One-factor model
|
1336.66
|
9
|
0.972
|
0.953
|
0.151
|
0.144, 0.158
|
Two-factor model
|
268.736
|
8
|
0.995
|
0.990
|
0.071
|
0.064, 0.079
|
Two-factor model(Lee et al.)
|
1270.999
|
8
|
0.973
|
0.950
|
0.156
|
0.149, 0.164
|
Two-factor model(Bessaha)
|
1041.832
|
8
|
0.978,
|
0.959
|
0.142
|
0.134, 0.149
|