Item finalization through correlation of preliminary items and iterative verification of reliability. Before analyzing the factor structure of the preliminary scale, I determined the degree of correlation between items by analyzing the overall correlation of each sub-factor and analyzed the internal consistency of the items. Analysis revealed that the correlation of 9 out of 55 items (11, 12, 13, 15, 22, 23, 24, 26, 55) was not statistically significant; thus, the nine items were removed. Afterwards, multiple factor analysis and item analysis were conducted to remove items with low factor loadings and reliability problems and that posed duplication problems in terms of content validity. Thus, 22 problematic items (7, 9, 10, 17, 18, 19, 20, 28, 29, 34, 35, 36, 41, 42, 43, 44, 45, 46, 47, 48, 49, 52) were eliminated based on these criteria. The final selection included 24 items.
Descriptive statistics of the scale items. Descriptive statistics were performed on the items to verify the validity of the final scale, and the results are shown in Table 1. Overall, the item scores were evenly distributed from one to five points with a mean at the three-point level. There were no items exceeding ± 3 in skewness, and kurtosis did not exceed ± 7, indicating that the items used in this study had no problem with normality.73, 74
Exploratory factor analysis. To identify the factor structure of the five factors of Coping Strategy, Cognitive Strategy, Ego Restriction, Interpersonal Functioning, and Frustration Tolerance that were theorized in process of the scale development, the principal axis factoring method was applied for EFA. The factorial rotation method used was the varimax technique, which is an orthogonal rotation method. The results of the EFA on the CESS showed that the KMO was .87, indicating a satisfactory selection of variables for factor analysis.75 In addition, Bartlett’s sphere formation test, which indicates the suitability of the factor analysis model, was χ2 = 4975.167 (p < .01). This indicates both that factor analysis is appropriate and that there is a common factor, meaning that the existing data for evaluating ego strength through game play attitude observation were suitable for factor analysis.
A criterion for the number of factors was based on the Kaiser standard (Eigenvalue > 1.0) and the scree test standard.76, 77 As shown in Figure 1 and the results from the EFA presented in Table 2, the items with eigenvalue of 1 or more among the 24 items of the scale were classified into five factors. Factor 1 accounted for 27.8% of the total variance with a factor loading ranging from .60 to .79, Factor 2 accounted for 13.3% with a factor loading ranging from .64 to .78, Factor 3 accounted for 8.6% with a factor loading ranging from .57 to .83, Factor 4 accounted for 6.8% with a factor loading of .60 to .82, and Factor 5 accounted for 4.6% with a factor loading ranging from .50 to .72. Thus, the five factors accounted for 61.2% of the total variance. Further, the five extracted factors were consistent with the theoretical concepts selected by the researcher at Stage 1 to measure ego strength through game play attitude observation.
Table 2
Exploratory Factor Analysis (N = 468)
|
Factor Loadings
|
|
Items
|
F1
|
F2
|
F3
|
F4
|
F5
|
Common Value
|
Coping Strategy
|
|
|
|
|
|
|
Complains of injustice when he/she loses
|
.79
|
.03
|
.15
|
−.03
|
.18
|
.67
|
Suspects the therapist of foul play when he/she loses.
|
.74
|
−.03
|
.18
|
.17
|
−.01
|
.61
|
Tries to invalidate the game by saying it was a “practice round” when he/she loses
|
.73
|
.04
|
.15
|
.00
|
.12
|
.57
|
Flips the board and throws the cards or game tools when he/she loses
|
.72
|
.15
|
.08
|
.10
|
.13
|
.58
|
Blames the win or loss on external factors
|
.69
|
.02
|
.09
|
.08
|
.32
|
.59
|
Cries or gets angry when he/she loses in the game a few times because he/she is unable to withstand the loss
|
.60
|
.13
|
.06
|
.14
|
.34
|
.52
|
Cognitive Strategy
|
|
|
|
|
|
|
Can appropriately use the strategies needed to win
|
−.05
|
.78
|
.21
|
.18
|
.14
|
.71
|
Remembers the game rules well
|
.23
|
.78
|
.15
|
−.06
|
−.09
|
.69
|
Verbally explains the rules of the game appropriately so that the therapist can understand them.
|
.12
|
.74
|
.00
|
.17
|
−.14
|
.61
|
Moves his/her piece by predicting the therapist’s next behavior
|
−.13
|
.72
|
.18
|
.21
|
.29
|
.69
|
Displays flexibility in changing strategy depending on the situation
|
−.12
|
.69
|
.18
|
.25
|
.27
|
.66
|
Is not distracted by surrounding stimuli and concentrates on the game.
|
.28
|
.64
|
.12
|
−.09
|
.11
|
.51
|
Ego Restriction
|
|
|
|
|
|
|
Says, “I won’t play,” and refuses to begin when listening to the game rules explained and they sound a bit complicated and difficult
|
.19
|
.25
|
.83
|
.00
|
.06
|
.79
|
Does not choose games that have slightly complicated rules
|
.06
|
.29
|
.83
|
.04
|
−.02
|
.77
|
Stops playing after a bit for games with slightly complicated rules.
|
.17
|
.31
|
.81
|
−.01
|
.09
|
.79
|
Tries to play alone for games he/she is not good at and with the therapist for games he/she is good at
|
.29
|
−.14
|
.57
|
.13
|
.19
|
.48
|
Interpersonal Functioning
|
|
|
|
|
|
|
Causes therapist to feel that he/she is interacting with the child when playing games with him/her.
|
.06
|
.09
|
.06
|
.82
|
.04
|
.69
|
Can appropriately ask the therapist for help when necessary
|
−.01
|
.13
|
−.07
|
.69
|
.07
|
.50
|
Enjoys playing games with the therapist.
|
.12
|
.04
|
.13
|
.64
|
−.27
|
.52
|
Is receptive to the therapist’s feedback when playing the game.
|
.39
|
.18
|
.03
|
.60
|
.18
|
.58
|
Frustration Tolerance
|
|
|
|
|
|
|
Expresses tension as a behavior when he/she seems to be losing
|
.26
|
.07
|
.01
|
−.03
|
.72
|
.59
|
Expresses excessive frustration in the style of having lost everything when he/she loses even once
|
.31
|
.03
|
.13
|
−.19
|
.66
|
.59
|
Waits patiently for the therapist without prodding when the therapist spends a bit of time thinking during his/her turn.
|
.44
|
.13
|
.03
|
.13
|
.52
|
.51
|
Does not stop playing and finishes the game even if he/she seems to be losing.
|
.22
|
.23
|
.25
|
.29
|
.50
|
.49
|
Eigenvalue
|
6.67
|
3.20
|
2.05
|
1.62
|
1.14
|
--
|
Explained variance %
|
27.80
|
13.34
|
8.55
|
6.76
|
4.75
|
--
|
Cumulative explained variance %
|
27.80
|
41.14
|
6.76
|
56.46
|
61.21
|
--
|
Confirmatory factor analysis. Next, CFA was conducted to examine whether the items and factors obtained through EFA fit the theoretical model. All scores were greater than .90 were considered to indicate a good fit, χ2 = 1172.504, p <.01, goodness of fit index indicator (GFI) = 0.944, Tucker-Lewis Index (TLI) = 0.945, and comparative fit index (CFI) = 0.965.78, 79 The root mean square error of approximation (RMSEA) = 0.048 was less than .08; based on the fitness index criterion, this factor structure can be evaluated as having good fit.80-83 In addition, the average variance extraction value was above .5, and the conceptual reliability value was above .7, verifying the validity of all sub-factors. The factor structure model of this scale is shown in Figure 1.
Correlation and reliability of game play attitude observation-based ego strength evaluation scale. The correlation between the whole scale and each sub-factor and the reliability are shown in Table 3. Each of five factors had a correlation ranging between .20 and .74 (for all correlations, p < .01) and was found to have a statistically significant positive correlation. The correlation between the overall scale and each sub-factor ranged between .51 and .74 (for all correlations, p < .01). Thus, all showed statistically significant positive correlation. These results show that each sub-factor measures a different domain of ego strength while simultaneously measuring one overarching super-concept. In addition, the Cronbach’s alphas were .84, .86, .85, .82, .70, and .70 for the whole scale, Coping Strategy, Cognitive Strategy, Ego Restriction, Interpersonal Functioning, and Frustration Tolerance, respectively. These results show that this scale has strong internal consistency among the items. In addition, the correlation coefficients between the sub-factors (rs ranging from .20 to .56) were lower than those between the whole scale and sub-factors (rs ranging from .51 to .74). These results show that the sub-factors constituting the scale developed in this study satisfy the criteria that they should be homogeneous yet somewhat independent.84
Table 3
Intercorrelations Between Factors and Internal Consistencies (N = 468)
|
F1
|
F2
|
F3
|
F4
|
F5
|
Total
|
Cronbach’s α
|
F1
|
1
|
|
|
|
|
|
.86
|
F2
|
.20**
|
1
|
|
|
|
|
.85
|
F3
|
.39**
|
.42**
|
1
|
|
|
|
.82
|
F4
|
.26**
|
.31**
|
.18**
|
1
|
|
|
.70
|
F5
|
.56**
|
.32**
|
.35**
|
.22**
|
1
|
|
.70
|
Total
|
.74**
|
.69**
|
.71**
|
.51**
|
.72**
|
1
|
.84
|
M
|
13.62
|
16.87
|
12.76
|
10.69
|
14.21
|
68.16
|
--
|
SD
|
4.89
|
4.89
|
3.94
|
2.63
|
3.43
|
13.56
|
--
|
Note. Mean (M); Standard Deviation (SD); F1 = Coping Strategy; F2 = Cognitive Strategy; F3 = Ego Restriction; F4 = Interpersonal Functioning; F5 = Frustration Tolerance; Total =Total Scale Score.
*p < .05, **p < .01
Stage 4: Verification of Criterion-Related Validity
Participants. Participants in Stage 4 included 25 play therapists and play therapist-supervisors who each had more than 10 years of clinical experience. To verify the criterion-related validity of this scale, 25 participants were each asked to select two children aged 7 to 9 years from their clinical cases, one should be judged to have strong ego strength, the other should be judged to have weak ego strength. These two children were either currently receiving counseling or had received counseling within the past six months. Of the 89 sessions with children that were analyzed, there were 24 game play therapy sessions of children evaluated to have strong ego strength and 63 game play therapy sessions of children evaluated to have weak ego strength.
In addition, 55 children aged between 7 and 9 years who received play therapy at two child counseling centers in Seoul area had their game playing attitudes analyzed to verify the scale’s concurrent validity. Children aged 7, 8, and 9 years were 30.9% (N = 17), 41.8% (N = 23), and 27.3% (N = 15), respectively, while 72.7% (N = 40) were boys and 27.3% (N = 15) were girls.
Measurements. The Rorschach test 85 measures the respondent’s cognitive style and psychodynamic construct based on the respondent’s response to ten inkblot cards. The evaluator directly assesses the child’s personality characteristics on the types of variables of the comprehensive system of this test, including resistance to stress, affect control, interpersonal skills, egocentricity, cognitive organization skills, copying deficit skill, presence of depression and perception disorder, excessive vigilance, and degree of compulsion.86 The Rorschach test provides an optimal opportunity to measure ego impairment because it induces the respondent to use cognitive, affective, and human or representational resources to organize an ambiguous and complex task.87 Thus, a number of researchers have verified the diagnostic efficacy and validity of the special indexes of the Rorschach CS,88, 89 and contributed to measuring ego strength by using certain variables of the Rorschach test or developing Rorschach Prognostic Scale (RPRS) and Ego Impairment Index (EII). 90, 91
Procedure and data analysis. The CESS was designed in a way such that higher scores are associated with weaker ego strength in order to improve clinical efficacy in detecting problematic behaviors. Thus, two positive items from F2: Cognitive Strategy, F4: Interpersonal Functioning, and F5: Frustration Tolerance were reverse scored. First, the participating children undergoing therapy were divided into strong ego strength and weak ego strength groups, and a t-test was performed to analyze differences in scores of the five factors of the CESS between the two groups in order to determine criterion-related validity of the CESS.
Next, concurrent validity of the CESS was determined by analyzing the correlations between the scores of the 5 factors of the CESS and scores of all variables of six special indexes of Rorschach CS (Perceptual Thinking Index; PTI, Depression Index; DEPI, Copying Deficit Index; CDI, Suicide Constellation; S-CON, Hypervigilance Index; HVI, Obsessive Style Index; OBS) and the seven sections of structural summary (Core Section, Ideation Section, Affect Section, Mediation Section, Processing Section, Interpersonal Section, and Self-Perception Section). Two clinical psychologists interpreted the Rorschach test of 55 children, and the inter-rater reliability was good, with .89.
Group Differences. As shown in Table 4, a significant difference between the two groups emerged through the analysis of the game play therapy sessions of children who were reported to have strong and weak ego strength to verify the criterion-related validity. As the CESS is designed in a way that higher scores represent weaker ego strength, a higher score on the CESS indicates weak ego strength. The children reported to have weak ego strength had significantly higher mean scores on the overall scale and in the sub-variables of Coping Strategy, Cognitive Strategy, Ego Restriction, Interpersonal Functioning, and Frustration Tolerance. This suggests that this scale can distinguish between groups with strong and with weak ego strength.
Table 4
Verification of Differences in Scores Between Strong and Weak Ego Strength Groups (N = 89)
|
Ego Strength
(N = 24)
|
Ego Weakness
(N = 63)
|
Difference
|
Factor
|
M (SD)
|
M (SD)
|
t
|
Coping Strategy
|
8.50 (3.62)
|
16.58 (16.59)
|
−6.41***
|
Cognitive Strategy
|
13.29 (3.86)
|
18.62 (5.32)
|
−4.47***
|
Ego Restriction
|
5.63 (2.60)
|
11.70 (5.15)
|
−5.51***
|
Interpersonal Functioning
|
8.00 (2.48)
|
10.67 (2.49)
|
−4.47***
|
Frustration Tolerance
|
8.04 (3.47)
|
12.24 (3.14)
|
−7.64***
|
Total Score
|
43.46 (12.05)
|
71.81 (14.90)
|
−9.16***
|
Note. ***p < .001
Concurrent Validity: Correlations between CESS and Rorschach. In order to determine concurrent validity of the CESS, we analyzed the correlations between the scores of the five factors of the CESS and scores of all variables in the seven sections of the comprehensive system of the Rorschach test (Core Section, Ideation Section, Affect Section, Mediation Section, Processing Section, Interpersonal Section, and Self Perception Section) and six special indexes (Perceptual Thinking Index, PTI; Depression Index, DEPI; Copying Deficit Index, CDI; Suicide Constellation, S-CON; Hypervigilance Index, HVI, and Obsessive Style Index, OBS).
The results are shown in Table 5. Mild to moderate correlations were obtained between the CESS factors and Rorschach variables. With regard to Rorschach index, the correlations between F5 (Frustration Tolerance) and F2 (Cognitive Strategy) factors and the CDI were significantly positive, r = .33 (p = .013) and r = .29 (p = .033), respectively. In addition, F3 (Ego Restriction) factor was significantly positively correlated to Depression index (DEPI), r =. 27 (p = .049). These results indicate that weaker ego strength is associated with lower social skills and coping skills or increased avoidance. The F1 (Coping Strategy) factor and the other CS variables, Afr (degree of interest in affective stimuli), S (white space response), and S-% (proportion of distorted form that involve use of white space), were also significantly positively correlated, r = .48 (p = .000), r = .41 (p = .002), and r = .31 (p = .023), respectively.
These results indicate that children who display greater number of inappropriate coping strategies are more likely to show emotional response, passively express frustration, and lose judgement ability so that they have a distorted perception of a situation when frustrated. F2 (Cognitive Strategy) factor was significantly negatively correlated with DQ+ (Developmental quality; assessment of a person’s ability to analyze and synthesize information) r = − .27 (p = .048), showing more use of inappropriate cognitive strategies is indicative of lower organizing skills using cognitive resources. F3 (Ego Restriction) factor was significantly correlated with number of response r = − .30 (p = .029), Pure C (number of pure color response) r = .35 (p = .009), a (number of flexibility responses) r =- .34, (p = .012), and Dd (unusual detail response) r =- .30, (p = .026). These results show children with high degree of ego restriction are unable to actively express the characteristics of their experience and are likely to have impulsive emotional response.
Table 5
Pearson correlations between CESS and Rorschach indices (N = 55)
|
F1
|
F2
|
F3
|
F4
|
F5
|
M
|
SD
|
CDI
|
.02
|
.29*
|
.15
|
.24
|
.33*
|
3.40
|
1.04
|
DEP1
|
.05
|
.08
|
.27*
|
.21
|
.14
|
3.24
|
1.15
|
PTI
|
.00
|
-.19
|
-.07
|
-.06
|
-.07
|
2.11
|
1.50
|
S-CON
|
.23
|
.03
|
.11
|
.16
|
.12
|
5.35
|
1.58
|
HVI
|
.17
|
.20
|
-.01
|
-.07
|
-.13
|
2.27
|
1.37
|
OBS
|
-.02
|
.08
|
-.14
|
-.11
|
-.08
|
0.73
|
0.71
|
Number of response
|
.05
|
-.03
|
-.30*
|
-14
|
-.09
|
19.24
|
8.60
|
Lambda
|
-.03
|
-.20
|
.14
|
-.04
|
.29*
|
2.30
|
3.00
|
Pure C
|
.21
|
.23
|
.35**
|
.26
|
.09
|
.45
|
.88
|
COP
|
-.15
|
-.17
|
-.18
|
-.27*
|
-.27*
|
.33
|
.70
|
Afr
|
.48***
|
.03
|
.04
|
.09
|
-.03
|
.44
|
.15
|
S
|
.41**
|
-.07
|
-.07
|
-.05
|
-.05
|
1.27
|
1.68
|
A
|
.04
|
-.18
|
-.34*
|
-.10
|
-.24
|
4.05
|
8.10
|
Dd
|
-.12
|
-.12
|
-30*
|
-.28*
|
-.06
|
2.64
|
3.66
|
DQ+
|
-.10
|
-.27*
|
-.23
|
-.20
|
-.14
|
3.45
|
3.34
|
S-%
|
.31*
|
.06
|
.12
|
.08
|
.07
|
0.06
|
0.11
|
Note. F1 = Coping Strategy; F2 = Cognitive Strategy; F3 = Ego Restriction; F4 = Interpersonal functioning; F5 = Frustration Tolerance; Copying Deficit Index (CDI); Depression Index (DEPI); Perceptual Thinking Index (PTI); Suicide Constellation (S-CON); Hypervigilance Index (HVI); Obsessive Style Index (OBS); the ratio of pure F response among the total responses (Lambda); pure color response (Pure C); Cooperative movement (COP); affect ratio (Afr); White space (S); active response (a); unusual detail response (Dd); Developmental quality (DQ+); White space distortion (S-%).
*p < .05, **p < .01, ***p < .001
F4 (Interpersonal functioning) factor was significantly correlated with COP (cooperative movement) r = − .27, (p ≤ .043) and Dd (unusual detail response) r = − .28 (p = .042). This demonstrates that children who show behaviors that are more inappropriate in interpersonal functioning are less able to positively interpret interactions with others and have poor sensitivity, such as difficulty detecting small details. F5 (Frustration Tolerance) factor was significantly correlated with Lambda (crude index of responsiveness) r = .27 (p = .036) and COP (cooperative movement) r = − .27 (p = .044). This shows that children with lower frustration tolerance are less able to utilize various resources in the environment and experience positive interaction with others.85