Psychometric Properties of the Emotional Processing Scale in Individuals with Psychiatric Symptoms and the Development of a Brief 15-Item Version

doi:10.21203/rs.3.rs-1216199/v1

Download PDF

Research Article

Psychometric Properties of the Emotional Processing Scale in Individuals with Psychiatric Symptoms and the Development of a Brief 15-Item Version

https://doi.org/10.21203/rs.3.rs-1216199/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Objective

The 25-item Emotional Processing Scale (EPS) is a measure of emotional processing. There is a scarcity of research about its factor structure, test-retest reliability, and validity in individuals with psychiatric symptoms.

Method

We administered the EPS-25 to a sample (N=512) of people with elevated psychiatric symptoms, using confirmatory factor analysis to evaluate three a priori models from previous research. We then evaluated discriminant and convergent validity against measures of alexithymia (Toronto Alexithymia Scale-20; TAS-20), depressive symptoms (Patient Health Questionaire-9; PHQ-9) and anxiety symptoms (Generalized Anxiety Disorder-7; GAD-7).

Results

None of the a priori models achieved acceptable fit, and subsequent exploratory factor analysis did not lead to a convincing factor solution for the 25 items. A 5-factor model did, however, achieve acceptable fit if we retained only 15 items (EPS-15) and this solution could be replicated in a holdout sample. Convergent and discriminant validity for EPS-15 was r=-.19-.46 vs. TAS-20, r=.07-.25 vs. PHQ-9, r=.29-.57 vs. GAD-7. Test-retest reliability was ICC=.73.

Conclusions

This study strengthens the case for the reliability and validity of the five factor Emotional Processing Scale but suggest that only 15 items should be retained. Future studies should further examine the reliability and validity of the EPS-15.

Psychology

emotional processing

factor analysis

validation study

somatic symptom disorder

After a stressful life event, a period of emotional processing follows (Rachman,, 1980). People try to “pick up the pieces” and “get over” or “come to terms” with what happened. Emotions are felt, expressed, and dealt with. Most often, the emotional disturbance of a stressful life event is absorbed, and life goes back to normal. However, sometimes, the emotional processing comes to a halt and there can be several signs of unprocessed emotional disturbances. People might start experiencing intrusions of the stressful life event or, in one way or another, and have difficulties controlling their emotions. Feelings can be overwhelming and be avoided or suppressed.

According to the emotional processing model (EPM), acknowledging emotions and finding an adaptive way of expressing them are necessary steps to cope with stressful life events (Baker et al., 2007, 2010). The EPM specifies in what way this process can be disrupted. In essence, avoiding or suppressing emotions will inhibit emotional processing and give rise to difficulties in emotional regulation.

To enable the study of emotional processing in accordance with the EPM, the Emotional Processing Scale (EPS) was developed. The original version of this scale comprised 38 items and 8 subfactors (Baker et al., 2007). This was later shortened to the 25-item Emotional processing scale (EPS-25) which has five subscales, believed to correspond to facets of emotional processing: avoidance, suppression, impoverished emotional experience, signs of unprocessed emotions, and unregulated emotion (Baker et al., 2010). First, avoidance is a way to avoid triggering emotions surrounding an event or situation. Second, suppression measures instances of suppressed feelings for example by trying not to show feelings. The third facet of emotional processing– impoverished emotional experience –captures aspects of “alexithymia”, when a person has difficulties identifying own emotions related to an event. The final two subscales of the EPS-25, signs of unprocessed emotions and unregulated emotions measure consequences of inadequate emotional processing. Signs of unprocessed emotions can, for example, manifest as nightmares whereas unregulated emotions can be expressed in temper tantrums.

Internal consistency and dimensionality

There have been several psychometric studies on the EPS-25, and studies generally report excellent internal consistency for the whole scale and fair to good internal consistency for each subscale (Baker et al., 2010; Gay et al., 2019; Górska, D. & Jasielska, A., 2010; Jasielska, A. & Górska, D., 2013; Kharamin et al., 2021; Lauriola et al., 2021; Orbegozo et al., 2018; Petermann, 2018).

In Baker et al.’s (2010) original article, the EPS-25 was administered to a mixed sample of medical patients, psychiatric patients, and healthy controls (N=690). Exploratory factor analysis revealed the 5-factor structure described above. Moreover, Gay et al. (2019) administered the EPS-25 to a combined sample (N=1176) of medical patients, hospitalised patients with bipolar disorder, city hall employees and students. Using exploratory factor analysis with the number of factors defined as 5 a priori revealed factor loadings relatively similar to those reported by Baker et al. (2010), though five items had cross-loadings over 0.30.

However, other attempts at replicating the original 5-factor solution have failed. Using two community samples (N=1172) Spaapen (2015) conducted a confirmatory factor analysis focusing on a 5-factor solution corresponding to that suggested by Baker et al. (2010). This factor structure did not achieve acceptable fit, either in its original form or based on a model with the 3 most problematic items dropped from the analysis. Orbegozo et al. (2018) administered the EPS-25 to school and university students (N=605) and investigated its factor structure using confirmatory factor analysis. Neither in this study, the original 5-factor model did achieve an acceptable fit. Moreover, neither Kharamin et al. (2021), who used a confirmatory factor analysis, and administered the EPS-25 to university students (N=1283) nor Lauriola et al. (2021), who administered the EPS-25 to a combined sample of gastrointestinal patients and healthy participants (N=696), could replicate the original five factor structure.

Since replication of the original findings of Baker et al. (2010) have proven difficult, several authors have proceeded in trying to find other ways to represent dimensionality. Spaapen (2015) investigated a two-factorial model, with suppression representing one factor, and the other subscales of EPS-25 representing another factor. Confirmatory factor analysis was however not able to establish a convincing model fit. Other authors have added a second-order latent “emotional processing” factor (i.e. a general emotional processing capacity factor) to the five subfactors, which have increased model fit in some studies (Lauriola et al., 2021; Kharamin et al., 2021). Another solution has been to reduce the number of items, and move items from one subfactor to another, to achieve adequate model fit (Orbegozo et al., 2018).

Taken together, despite difficulties in replicating Baker et al.’s (2010) original findings, most previous studies speak for a five factor model, either with a second order latent “emotional processing” factor (Lauriola et al., 2021; Orbegozo et al., 2018; Kharamin et al., 2021) or not (Gay et al., 2019; Baker et al., 2o10). However, in order to establish a five factor model, revisions have been made in some studies; such as reducing items (Orbegozo et al., 2018; Spaapen, 2015).

Convergent and discriminant validity

Convergent and discriminant validity concerns the magnitude of correlations between a measure, such as the EPS, and other measures of relevance. Because EPS is a measure of dysfunctional emotional processing, it could be expected to correlate quite consistently with measures of similar constructs such as alexithymia (i.e., difficulties identifying and expression emotions) but should show less overlap with constructs not directly part of emotional processing (such as depression).

The convergent validity of EPS-25 has been studied in relation to concept such as emotional control (Baker et al., 2007) and emotional regulation (Orbegozo et al., 2018; Gay et al., 2019), Moreover, the concepts of emotional processing and alexithymia are believed to partly overlap and hence, in certain aspects, have convergent validity (Baker et al., 2007; 2010). Especially the subfactor Impoverished emotional experience in the EPS-25 has been proposed (Baker et al., 2007; 2010) to be like the alexithymic feature of having difficulties identifying own feelings (measured by the Toronto Alexithymia Scale-20, factor 1). Although theoretically sound, only a rather weak concurrent validity (r=.35) with the TAS-20 subscale Identifying feelings and the EPS-25 subscale Impoverished emotional experience has been found (Gay et al., 2019; 2021).In addition, the capacity for EPS-25 to show an adequate discriminant validity have been mixed, pointing to an inflated relationship between measures of emotional processing and negative affect. EPS-25 has in three studies been found to correlate quite largewith measures of anxiety (r=.47-59) and depressive symptoms (r=.48-63) (Orbegozo et al., 2018; Gay et al., 2019; Kharamin et al., 2021). Emotional processing is also believed to have discriminant validity with facets of the alexithymia construct (Baker et al., 2007), specifically what is coined external oriented thinking (measured by the Toronto Alexithymia Scale-20, factor 3). In line with this, EPS-25-total has been shown to be uncorrelated with this subscale (Gay et al., 2019; 2021).

Test-retest reliability

Only two studies have investigated EPS-25 test-retest reliability. Using a convenience sample of 17 healthy individuals, test–retest reliability was assessed over a 4- to 6-week period (Baker et al., 2010). The Pearson's test–retest correlation coefficient obtained for the entire scale was .74. Moreover, 80 participants were randomly selected from a cohort of 1285 students demonstrating a Pearson's test–retest correlation coefficient for the entire scale of .91 (retest administration made at 4 weeks) (Kharamin et al., 2021).

Aims of the study with hypotheses

Planned aims

The overarching aim of this study was to conduct a structural validation of the EPS-25 in a sample of patients with elevated psychiatric symptoms. We hypothesized that the EPS-25 would follow either a 5-factorial with or without a second-order latent factor or possibly a 2-factorial model. The 2-factorial model would follow Spaapen (2015) description of emotional processing consisting of 2. The 5-factorial model would follow Baker et al.’s, 2010 description of emotional processing consisting of five different facets or subscales. Having a second-order factor would mean that, in addition to the 5 factors, a higher-order or general emotional processing capacity would be found.

We hypothesized that internal consistency for both the EPS would be good (α ≥ 0.80) for the total scale and at least fair for the subscales (α ≥ 0.60). Further, convergent validity of the EPS and the Toronto-Alexithymia Scale-20 (TAS-20) was believed to be quite high (hypothesis: r≈.50-.75 vs. alexithymia – total score and score for TAS-20 factor 1, Identifying feelings). For measures of discriminant validity, anxiety (GAD-7) and depression (PHQ-9) was calculated and believed to be lower than the EPS relationship with TAS-20 (hypothesis: r≈.25-.50 vs. depression, r≈.25-.50 vs. anxiety). Moreover, the EPS was not believed to be correlated with external oriented thinking style from TAS-20, factor 3. A last aim of this study was to evaluate EPS test-retest reliability which was hypothesized to have an adequate test-retest reliability (i.e. ICC≥0.60) over approximately 1 week.

Revised aims

As results of the EPS-25 became clear, a further aim of this study was to investigate whether a shorter version of the EPS-25, the EPS-15, might be created and would be psychometrically sound.

Data

The structural validation in this study was based on baseline data, i.e., data collected priori to any clinical intervention, from four clinical trials of internet-delivered psychodynamic treatment (Johansson et al., 2013, 2017; Maroti et al., 2021; in preparation). First, one randomized controlled trial in which affect-focused therapy was compared to basic support from a clinician for participants with an anxiety disorder or depression (Johansson et al., 2013). Second, one randomized controlled trial where affect-focused therapy was compared to a waiting-list for participants with social anxiety disorder (Johansson et al., 2017). Third, one prospective cohort trial of emotional awareness and expression therapy for participants with somatic symptom disorder (Maroti et al., 2021). Fourth, one randomized controlled trial where emotional awareness and expression therapy was compared against a waiting-list for participants with somatic symptom disorder (Maroti et al., in preparation). Data are available on request by the corresponding author. The research was performed in accordance with the Declaration of Helsinki, all participants (N=512) provided informed consent and all four clinical trials were conducted in accordance with relevant regulations and approved by the appropriate regulatory authorities: (Regional Ethics Board of Linköping: 2011/400-31, 2013/361-31; Swedish Ethical Review Authority: 2019-03317, 2020-03490). ClinicalTrials.gov identifiers are: NCT01532219, NCT02105259, NCT04122846 and NCT04751825.

For discriminant and convergent validity, data from one of the four trials were used where Emotional Awareness and Expression Therapy for somatic symptom disorder was compared to a waiting-list. The reason for this was that this trial was the only one with complete data of EPS, TAS-20, PHQ-9 and GAD-7. That is, none of the other trials used all measures needed for discriminant and convergent validity analysis.

For calculating test-retest reliability, data from a published trial of EAET were used (Maroti et al., 2021; data of EPS not reported). This was the only trial (of the four described) which had an adequate number of days (Range: 5-12 days) between the first administration and a second administration of the EPS. Fifty-one participants completed the EPS at two time points prior to treatment.

Participants and recruitment

Patients with mixed elevated scores of anxiety and depression were recruited as part of two completed internet delivered treatment studies (Johansson et al., 2013; 2017). Patients with elevated somatic symptoms were recruited as part of a completed (Maroti et al., 2021) and ongoing (Maroti et al., in preparation) internet-delivered treatment study of Emotional Awareness and Expression Therapy for somatic symptom disorder. In all studies, adult participants were recruited from the community by advertisement and were enrolled using a safe internet platform. Main common exclusion criteria were: no other major psychiatric conditions where treatment provided by psychiatric outpatient care would be more appropriate (e.g., psychosis, suicidal ideation). Comorbidities were allowed. For further details of recruitment procedures, see Johansson et al., 2013; 2017 and Maroti et al., 2021; in preparation.

Procedure

The Emotional Processing Scale (EPS-25) has 25 items where each item uses a 10-point visual analogue rating scale, from 0 (= completely disagree) to 9 (= completely agree). The main scale score is calculated by dividing the total item sum by 25. This procedure is identical for subscales (avoidance, suppression, impoverished emotional experience, signs of unprocessed emotions and unregulated emotions) although the total is divided by five. The EPS was translated from English to Swedish in accordance with Brislin (1970), where the instrument is first translated from Swedish and English back and forth until a satisfactory translation is reached (Brislin, 1970). This was done by three people with excellent knowledge of Swedish and English (e.g., having English as their native language). In addition to EPS, we administered other measures used in this study to investigate convergent and divergent validity:

The Toronto Alexithymia Scale (TAS-20) is a 20-item measure of alexithymic traits (Bagby, Parker, et al., 1994a; Bagby, Taylor, et al., 1994b). The instrument includes three subscales: (1) Difficulty Identifying Feelings (DIF), (2) Difficulty Describing Feelings (DDF) and (3) Externally Oriented Thinking (EOT). Each item is rated on a 5-point Likert-type scale ranging from 1 (strongly disagree) to 5 (strongly agree), and the total score can range from 20 to 100. The TAS-20 has shown both good internal consistency and test–retest reliability in the Swedish population (Simonsson-Sarnecki et al., 2000).

The Patient Health Questionnaire-9 (PHQ-9; Kroenke et al., 2010) is a measure of depressive symptom severity. It consists of nine items rated 0 to 3, and total scores range from 0 to 27. The PHQ-9 has good psychometric properties, including an internal consistency in the range of Cronbach’s α = 0.86 – 0.89 (Kroenke et al., 2010).

The Generalized Anxiety Disorder-7 scale (GAD-7; Spitzer et al., 2006) is a measure of anxiety symptom severity. It consists of seven items rated 0 to 3, and total scores range from 0 to 21. Internal consistency is excellent (Cronbach’s α = 0.92) (Kroenke et al., 2010).

Participants accessed self-report questionnaires via a personal account on the study web platform. The questionnaires could be completed at any time of the day and using any conventional web browser, for example using a laptop, tablet or smart phone. Traffic with the web platform was encrypted, and all studies proceeded in accordance with relevant data management and privacy legislation.

Planned analysis

Within a confirmatory factor analytic (CFA) framework, using all available data (N=512), we tested three different possible factor solutions in the R 4.1.0 (R Core Team, 2016) with lavaan 0.6-8: 1) a 5-factor model corresponding to the original solution presented by Baker et al. (2010); 2) a 5-factor model with a second order “emotional processing” latent variable as found to be adequate in previous studies (Kharmin et al., 2021; Lauriola et al., 2021); and 3) a 2-factor model with suppression and other factors as discussed by Spaapen (2015). Criteria for good model fit were: CFI and TLI 0.90 or ideally 0.95, RMSEA and SRMR < 0.08, lowest possible AIC and BIC (Hu & Bentler, 1999).

Using Jamovi (The jamovi project, 2021), we analysed EPS internal consistency in terms of Cronbach’s α, investigated convergent and discriminant validity based on Pearson correlations and estimated test-retest reliability based on the intraclass correlation coefficient (ICC). For the α statistic, if scales are of typical length, values ≥0.90 are commonly regarded excellent, ≥0.80 good, and ≥0.70 acceptable. Importantly however, in short scales, α also decreases substantially with the number of items such that, for example, if inter-item correlations are moderate, a transition from 5 to 3 items could be expected to result in a lowering of α of around 0.10-0.15 units. For the r statistic, values around 0.50 are commonly regarded as indicative of a strong correlation (i.e., a large effect), 0.30 moderate and 0.10 weak (i.e., a small effect) (Cohen, 1992). For the ICC, values ≥0.75 are commonly regarded excellent, ≥0.60 good, ≥0.40 fair and <0.40 poor (Cicchetti, D., 1994) .

Factor Structure

Confirmatory factor analysis (CFA) of EPS-25

Because kurtosis was high for several items, and some items had many zero scores, models were fit using maximum likelihood estimation with robust (Huber-White) standard errors and a scaled test statistic. In the CFA using all data, none of the three a priori models of EPS-25 achieved adequate fit (see Table 1). Theoretically sound changes in accordance with modification indices including the removal items (6, 8, 14, 17, 23) and the specification of reasonable residual covariance (for example 18, 19) also did not result in acceptable model fit.

Table 1

Fit indices and other key dimensionality parameters derived from factor analysis of the Emotional processing scale
Framework	Model	Data	Items	χ²	χ² / df	P	CFI	TLI	RMSEA (90% CI)	SRMR	AIC	BIC	Cross-loadings or as indicated	All loadings <0.4	Factor corr.
Phase 1: test of a priori models
CFA	A priori 5F two-tier	Total	25	1060	3.9	<.001	0.84	0.82	0.082 (0.077, 0.087)	0.079	57211.8	57443.6	MOI: 6, 14, 17, 19, 23	4, 14	0.66-0.96^a
CFA	A priori 5F	Total	25	1016	3.8	<.001	0.85	0.83	0.080 (0.075, 0.085)	0.074	57166.8	57419.6	MOI: 6, 14, 17, 18, 19, 23	4, 14	0.42-0.84
CFA	A priori 2F	Total	25	1323	4.8	<.001	0.78	0.76	0.094 (0.089, 0.099)	0.085	57516.0	57730.9	MOI: 5, 6, 14	4, 14	0.63
Phase 2: exploratory modeling
EFA	1F	Training	25/23^c	1078	3.9	<.001		0.67	0.107 (0.100, 0.114)			-447	Not applicable	4, 14	Not applicable
EFA	Free 2F^b	Training	25/21^c	667	2.7	<.001		0.81	0.080 (0.073, 0.088)			-725	≥0.4: 14	4, 5, 6, 12	0.66
EFA	Free 5F	Training	25/22^c	300	1.6	<.001		0.93	0.049 (0.039, 0.060)			-725	≥0.4: 6, 8, 19, 23	9, 10, 14	0.14-0.74
CFA	EPS-15 5F^d	Training	15	125	1.6	.001	0.96	0.95	0.049 (0.031, 0.065)	0.046	17875.0	18016.8	MOI: 2, 3, 5, 8, 9, 21, 22, 24, 25	None	0.45-0.75
Phase 3: validation
CFA	EPS-15 5F^d	Validation	15	180	2.2	<.001	0.92	0.89	0.075 (0.060, 0.090)	0.053	16987.9	17127.7	MOI: 5, 16	None	0.36-0.83
Robust fit indices from confirmatory factor analysis. Note that because these models are fitted on different data (the total, training, and validation sample) all values are not directly comparable. Note also that the 2 and 5 factor solutions derived from exploratory factor analysis where all items are allowed to freely load on all factors (that is, cross-loadings are estimated freely over all factors) do not necessarily correspond to other published factor solutions such as those of Baker et al. (2010) or Lauriola et al. (2021). Due to software limitations, fewer fit indices are provided for the EFA models. 2F, two-factor; 5F, five-factor; AIC, Akaike information criterion; BIC, Bayesian information criterion; CFI, comparative fit index; EPS-15, 15-item version of the emotional processing scale; MOI, 20 largest modification indices (1-df); RMSEA, root mean square error of approximation; SRMR, standardized root mean square residual; TLI, Tucker Lewis index.
^a Loadings on the latent “emotional processing” factor.
^b Arguably the most promising model according to the scree plot, with a clear increase in eigenvalue and deviation from factors derived from simulated data occurring between factor 3 and 2 (see figure 1).
^c Note that while all 25 items were included in the analysis, none of the factor solutions derived from EFA resulted in all 25 items having factor loadings of at least 0.4 on at least one factor. For example, in the 5-factor solution derived from EFA, items 9, 10, and 14 did not load 0.4 or higher on any factor, which means that this was in effect a 22-item solution.
^d This is the final 15-item factor solution, reached primarily via stepwise modification of the original CFA a priori 5F (non-two-tier) model. See the main text for details.

Table 2

Comparison between blinded theoretical judgment (by DM and RJ), initial empirical suggestion of problematic items, and the final EPS-15.
			Theoretical judgment	Sum of EPS-25 modification indices indicative of potential cross-loadings^a	EPS-15 final item selection	Factor loadings in validation sample
Subscale	#	Items paraphrased
Suppression	1	Smothered feelings	Yes	0	X	0.78
	6	Could not express feelings	No	145
	11	Kept quiet about feelings	Borderline	13
	16	Bottled up emotions	Yes	0	X	0.85
	21	Tried not to show feelings	Yes	0	X	0.76
Signs of unprocessed Emotions	2	Unwanted feelings kept intruding	Yes	0	X	0.77
	7	Emotional reactions lasted more than a day	Borderline/Yes	0	X	0.71
	12	Repeatedly experienced the same emotion	Borderline/Yes	13
	17	Overwhelmed by emotions	Borderline	73
	22	Thinking about same emotion again and again	Yes	0	X	0.74
Unregulated emotions	3	When upset difficult to control what I said	Yes	0	X	0.68
	8	Reacted too much to what people said or did	Yes	11	X	0.79
	13	Wanted to get own back on someone	Yes	0	X	0.51
	18	Felt urge to smash something	Yes	22
	23	Hard to wind down	Borderline	78
Avoidance	4	Tried to avoid things that might make me upset	Yes	0	X	0.40
	9	Talking about negative feelings made them worse	Borderline	0	X	0.67
	14	Tried to talk only about pleasant things	Borderline/Yes	50
	19	Could not tolerate unpleasant feelings	No	12
	24	Avoided looking at unpleasant things	Borderline/Yes	0	X	0.69
Impoverished emotional experience (“alexithymia”)	5	Emotions felt blunt/dull	Borderline	11	X	0.47
	10	Feelings did not seem to belong to me	No	0
	15	Hard to work out if I felt ill or emotional	Yes	0	X	0.61
	20	Seemed to be a big blank in feelings	Borderline	0
	25	Strong feelings but not sure if emotions	Borderline/Yes	0	X	0.73
EPS-15: Emotional processing Scale. Yes: endorsed as an item typical what the subscale is supposed to measure. No: not endorsed. Borderline: neither typical nor atypical.
^a This is the sum of all modification indices pertaining to item-factor cross-loadings for each item, as based on the top 30 modification indices for the a priori 5-factor solution for the EPS-25 when fitted on the training data. Note that while there is considerable overlap between blinded theoretical judgments, these modification indices, and the final EPS-15, the reduction of the number of items from 25 to 15 was an iterative process where modification indices where examined for several intermediate scale forms, which for example is why we ultimately decided to include item 5 over items 10 and 20 in the EPS-15.

Internal consistency of EPS-25

Cronbach’s alpha was excellent for the EPS-25 sum scale (α = .92), good for the avoidance subscale (α = .87) and impoverished emotional experience subscale (α = .83), and acceptable (α = .67-.75) for the remaining subscales.

Convergent and discriminant validity of EPS-25

As seen in Table 3, the EPS-25-total showed strong correlations with anxiety (r=.66) and depression (r=.59). A moderate correlation of Impoverished emotional experience and the alexithymia factor Identifying feelings (TAS-20, factor 1) was found (r=.35) but also weak correlation (r=.21) with external oriented thinking style (TAS-20, factor 3).

Table 3

Correlations between the 25-item emotional processing scale total and subscale scores and the TAS-20, PHQ-9 and GAD-7 (N=74, Only data from Maroti et al., in prep).
	EPS-25 total	Avoidance	Suppression	Impoverished emotional experience	Signs of unprocessed emotions	Unregulated emotions
TAS-20
Total	.57***	.57***	.29***	.35***	.46***	.61***
Describing feelings	.49***	.53***	.20**	.27***	.40***	.56***
Identifying feelings	.65***	.52***	.45***	.48***	.53***	.59***
Externally oriented thinking	.21**	.33***	.02	.04	.15*	.31***
GAD-7	.66***	.40***	.57***	.61***	.54***	.50***
PHQ-9	.59***	.43***	.44***	.48***	.47***	.49***
*<0.001 <0.01 *<0.05. TAS-20= Toronto Alexithymia Scale. EPS-15=Emotional Processing Scale. GAD-7=Generalized Anxiety Disorder. PHQ-9: Patient Health Questionnaire.

Test-retest reliability

The test-retest reliability was excellent for EPS-25 (ICC=0.76).

(Range: 5-12 days with M=8.06, SD=1.35)

Revised analysis: exploratory factor analysis and the development of a short version of the EPS: the EPS-15

Because we could not replicate any of the a priori factor structures for the 25-item version of EPS using confirmatory factor analysis, we made an attempt to find a more suitable factor solution using exploratory factor analysis. Moreover, although the EPS-25 showed good internal consistency and test-retest reliability, discriminatory validity was unsatisfactory. EPS-25 showed both an inflated relationship with depression and anxiety and had a correlation with facets of the alexithymia construct (TAS-20, factor 3: External oriented thinking) not in line with suggestions from Baker et al., (2007).

For the purpose of finding and validating a more suitable factor solution, the sample was subsequently split into training (n=262) and validation (n=250) subsamples by randomization. We deemed these sample sizes adequate for factor analysis considering that they were close to the common recommendation of 300 (Field,, 2018) and we expected communalities to be at least moderate (Guadagnoli & Velicer, 1988). We proceeded to conduct an exploratory factor analysis (EFA) based on principal factor axis factoring with promax (oblique) rotation in Jamovi (The Jamovi project, 2021), with the intention of finding an empirically and theoretically sound factor solution for the data. In these analyses, we explored 1-, 2-, and 5-factor solutions as informed by the scree plot (see Figure 1) and our theoretical understanding of the scale and the emotional processing model. We wanted to achieve distinct factors as characterized by factor loadings of ≥0.30 or ideally ≥0.40, with few or ideally no substantial cross-loadings, few or ideally no items not belonging to no factor, and at least three items loading on each factor.

Exploratory factor analysis (EFA)

The training data set was confirmed to be suitable for factor analysis (Barlett’s test p < .001; KMO = 0.91). We found the scree plot (Figure 1) inconclusive; the knee being indicative of 1, or more probably 2, factors while parallel analysis resulted in weaker eigenvalues up to factor 5 even though the difference was small for factors 3-5. However, none of the freely estimated 1-, 2-, or 5-factor solutions for the EPS-25 could achieve acceptable fit with distinct factors, meaning that each of all 25 items had a factor loadings of at least 0.40 with minimal cross-factor loadings (see Table 1).

Development and dimensionality of the 15-item Emotional processing scale

As neither confirmatory nor exploratory factor analysis resulted in an acceptable factor solution for the EPS-25 we aimed at developing a shorter 15-item scale – the EPS-15 - with a more distinct factor structure. The purpose was to identify a subset of the EPS-25 items that would allow for stronger dimensionality in terms of model fit and distinct yet correlated factors. To achieve this goal, we based the item selection process on the best fitting 25-item CFA model (this had 5 factors corresponding to the conventional subscale scoring), and the stepwise deletion of items, and addition of covariance if theoretically feasible, based on modification indices and our theoretical understanding (see Table 2), until exactly 3 items remained for each of the 5 factors. We subsequently validated this model in the validation data.

Statistical considerations

Based on modification indices for the 25-item CFA training data 5-factor solution, item 6 (“Could not express feelings”) was moved to the impoverished emotion factor. The following items were then removed step by step: 23, 17, 8, 14, 5, 6, 20, 19, 11, and 12. We found the correlation between the unregulated emotions and suppression factors to be unsatisfactory (r=0.31, i.e., clearly lower than 0.40) and therefore replaced item 18 (“Felt urge to smash something”) with item 8 (“Reacted too much to what people said or did”) to increase this correlation to an acceptable r=0.45. The resulting five-factor model using 15 items (three per factor) achieved acceptable model fit in terms of the RMSEA (0.050, [90% CI 0.033, 0.066]), SRMR (0.046), CFI (0.96), and TLI (0.95).

Theoretical considerations

To reach the best suiting 15 items of the 25 items available, the items were also scrutinized for adequate content validity by D.M and R.J, who, at that time, did not know of what 15 item the structural validation had suggested (see Table 2). The items were deemed either fully indicative of its subfactor (coded as “yes”), only partly so (coded as “borderline”) or not an adequate description of content validity (coded as “no”). Following these considerations, one additional change of the EPS-15 was made as we did not regard item 10 (“My feelings did not seem to belong to me”) as a convincing example of impoverished emotional experience understood as alexithymia (Sifneos, 1991) and therefore instead reintroduced item 5 (“My emotions felt blunt/dull”; see Table 2).

The resulting final 15-item five-factor model (from both statistical and theoretical considerations) showed improved model fit in the training data and achieved acceptable, though not ideal, fit in the validation data (see Table 1). All factor correlations and factor loadings were 0.40 or higher in the training data and remained so in the validation data, with the exception of the correlation between the unregulated emotions and suppression factors that dropped to 0.36 in the validation.

Internal consistency of the EPS-15

As shown in Table 4, Cronbach’s alpha was good for the EPS-15 sum scale (α = .87) and the suppression subscale (α = .83), and acceptable (α = .62-.76) for the remaining subscales.

Table 4

Internal consistency for the Swedish validation EPS-25 and EPS-15 (n=51).
	Cronbach’s alpha
	EPS-25.	EPS-15.
EPS total	.92	.87
Avoidance	.87	.62
Suppression	.75	.83
Impoverished emotional experience	.83	.64
Signs of unprocessed emotions	.67	.76
Unregulated emotions	.79	.69

Convergent and discriminant validity of EPS-15

As seen in Table 5, the EPS-15-total had a strong correlation with anxiety (r= .57). but a weak correlation with measures of depressive symptoms (r=.25). Moreover, a moderate correlation of Impoverished emotional experience and the alexithymia factor Identifying feelings (TAS-20, factor 1) was found (r=.46), whereas there was no significant correlation with external oriented thinking style (TAS-20, factor 3).

Table 5

Correlations between the 15-item emotional processing scale total and subscale scores and the TAS-20, PHQ-9 and GAD-7 (N=74, Only data from Maroti et al., in prep).
	EPS-15 total	Avoidance	Suppression	Impoverished emotional experience	Signs of unprocessed emotions	Unregulated emotions
TAS-20
Total	.34**	.24*	.39***	.41***	.13	.13
Describing feelings	.30*	.23	.45***	.30**	.09	.05
Identifying feelings	.47***	.31**	.35 **	.46***	.33**	.30**
Externally oriented thinking	-.04	-.03	.09	.13	-.19	-.11
GAD-7	.57***	.50***	.29*	.34**	.49***	.47***
PHQ-9	.25*	.27*	.32**	.19	.08	.07
*<0.001 <0.01 *<0.05. TAS-20= Toronto Alexithymia Scale. EPS-15=Emotional Processing Scale. GAD-7=Generalized Anxiety Disorder. PHQ-9: Patient Health Questionnaire.

Test-retest reliability

The test-retest reliability was good for the EPS-15 (ICC=0.73).

(Range: 5-12 days with M=8.06, SD=1.35)

Based on data from 512 individuals with elevated psychiatric symptoms, we could not find a satisfactory factor solution for the 25-item Emotional Processing Scale. This led us to develop a brief 15-item version of the scale, the EPS-15, for which we found an acceptable 5-factor solution that we could validate using a split sample strategy. EPS-15 had good internal consistency and test-retest reliability and demonstrated discriminant validity from the construct of depressive symptoms, although to a lesser degree of anxiety symptoms.

Inconsistent findings pertaining to the EPS-25

Evidence pertaining to the factor structure of the EPS-25 was inconsistent. On the one hand, the training data scree plot (see figure 1) appeared to be indicative of 1 or probably 2 factors (suppression vs. other). On the other hand, in parallel analysis higher eigenvalues were obtained up to the fifth factor, and five-factor solutions were most promising in terms of model fit under both CFA and EFA. Most previous studies appear to speak for some type of five-factor model, either with a second order latent “emotional processing” factor (Lauriola et al., 2021; Orbegozo et al., 2018; Kharamin et al., 2021) or not (Gay et al., 2019; Baker et al., 2o10). There are several potential explanations for the difficulties we encountered in replicating these five-factor solutions. In CFA, as suggested by Lauriola et al., (2021), poor fit may have been a result of this type of model not allowing for cross-loadings over factors in the same manner as EFA as used in the original publication (Baker et al., 2010). This said, undue reliance on cross-loadings for model fit could also be indicative of poorly defined factors, the existence of weak main factor loadings in combination with strong cross-loadings speak against scoring of the five conventional subscales, and the brief EPS-15 did indeed achieve acceptable fit under CFA. Importantly, in our data, the EPS-25 5-factor EFA model where item factor loadings were estimated freely was also not satisfactory, especially as it had a pattern of factor loadings that was clearly inconsistent with the conventional scoring of the EPS-25 subscales; items 6, 8, 19, 23 exhibiting cross-loadings and items 9, 10, 14 not belonging to any factor. Another potential explanation is that the unexpected outcome may be somehow related to sampling, for example the fact that that few previous studies of the EPS-25 have focused as heavily on individuals with high levels of psychiatric symptoms.

Properties and potential advantages of the EPS-15

The factor structure of the EPS-15 brief scale appeared to be replicable over the testing and validation samples. Considering that all item factor loadings were 0.40 or higher, and acceptable model fit was achieved in the CFA framework without the need to specify cross-loadings (that is, each item loaded on its intended factor only), the factor solution appeared to be in support of the scoring of five subscales.

As already stated, the EPS-15 demonstrated discriminant validity from the construct of depressive symptoms, although to a lesser degree of anxiety symptoms. It should be noted that in previous studies, the EPS-25 has not shown adequate discriminant validity from either depressive or anxiety symptoms (Orbegozo et al., 2018; Gay et al., 2019; Kharamin et al., 2021) and, in this study, the EPS-25 had an inflated relationship not only with anxiety symptoms but with depressive symptoms. Moreover, it has been suggested (Baker et al., 2007) that the EPS should not overlap with Toronto Alexithymia Scale, External oriented thinking style (TAS-20 factor 3), but in previous studies, subscales of the EPS-25 have been found to correlate significantly with TAS-factor 3 (Gay et al., 2019; 2021), although the correlation was weak. Moreover, two of the subscales of the EPS-25 in this study, did show moderate correlation with External oriented thinking style (TAS-20 factor 3). This was not the case in this study for EPS-15 (see Table 4), further strengthening EPS-15 discriminant validity.

In the only two previous studies that have researched EPS-25 test-retest reliability, this has found to be good to excellent (Baker et al., 2010; Kharamin et al., 2021). In this study, both EPS-25 and EPS-15 demonstrated a good test-retest reliability.

Compared to the EPS-25, the EPS-15 may be easier to administer and complete, especially when used in combination with many other scales (such as in routine care screening batteries), when space is limited (such as in epidemiological research) and for the purpose of repeated measurements (such as during treatment or as part of rigorous research designs). Moreover, without losing any of the important psychometric strengths of the EPS-25 (such as internal consistency, test-retest reliability), EPS-15 were better able to discriminate emotional processing from depressive symptoms and facets of the alexithymia construct.

Limitations

There are limitations of this study. In psychometric terms, the test-retest and discriminatory and convergent validation analyses were conducted on a small sample. Also, the use of randomization to form the testing and validation subsamples did not result in as stringent of a validation as a true replication in data from an entirely new sample.

Participants self-selected to take part in this study (or more precisely: to internet-delivered emotion-focused treatment), and it is not clear if this population is like the ones encountered in clinical practice, limiting the conclusions that can be drawn for example about test-retest reliability. Moreover, the sample used for studying concurrent and discriminant validity were all diagnosed with somatic symptom disorder, and generalization to other samples, including healthy people and those with other psychiatric conditions, is limited.

Overall discussion and future studies

Despite the fruitful modelling and measurement of emotional processing, considerable challenges remain, and many important questions require further investigation.

Further structural validation studies of the EPS-15 comparing different populations (i.e. comparing psychiatric to a healthy population) would be of interest. EPS-15 convergent validity should further be clarified, by using studies with a measure closely overlapping with emotional processing, such as emotional awareness (measured by Level of Emotional Awareness Scale) or emotional regulation (measured by Difficulties of Emotional Regulation Scale).

In conclusion, the EPS-15 is a promising short form questionnaire to be used in clinical and treatment studies, although both structural and other parts of validity and reliability needs to be further delineated.

Author contribution: DM, EA and RJ made substantial contributions to conception of the work. DM wrote the first draft where especially EA, but also all other authors (BL, ML, RJ & GA), made substantial contributions to the revision of the manuscript. EA had the leading role of the formal analysis, but DM and RJ contributed. GA and RJ made a substantial contribution of the acquisition of data.

Funding: This study was founded by Söderström-Königska foundation and Karolinska Institute´s research grant.

Conflict of interest: The authors declare no conflict of interest. Publication of research is approved by Dr Roger Baker, developer of EPS-25 and Hogrefe Publishing Group, owner of EPS-25.

Ethical approvals: The research was performed in accordance with the Declaration of Helsinki, all participants (N=512) provided informed consent and all four clinical trials were conducted in accordance with relevant regulations and approved by the appropriate regulatory authorities: (Regional Ethics Board of Linköping: 2011/400-31, 2013/361-31; Swedish Ethical Review Authority: 2019-03317, 2020-03490). ClinicalTrials.gov identifiers are: NCT01532219, NCT02105259, NCT04122846 and NCT04751825.

Bagby, R. M., Parker, J. D. A., & Taylor, G. J. (1994). The twenty-item Toronto Alexithymia scale—I. Item selection and cross-validation of the factor structure. Journal of Psychosomatic Research, 38(1), 23–32. https://doi.org/10.1016/0022-3999(94)90005-1
Bagby, R. M., Taylor, G. J., & Parker, J. D. A. (1994). The twenty-item Toronto Alexithymia scale—II. Convergent, discriminant, and concurrent validity. Journal of Psychosomatic Research, 38(1), 33–40. https://doi.org/10.1016/0022-3999(94)90006-X
Baker, R., Thomas, S., Thomas, P. W., Gower, P., Santonastaso, M., & Whittlesea, A. (2010). The Emotional Processing Scale: Scale refinement and abridgement (EPS-25). Journal of Psychosomatic Research, 68(1), 83–88. https://doi.org/10.1016/j.jpsychores.2009.07.007
Baker, R., Thomas, S., Thomas, P. W., & Owens, M. (2007). Development of an emotional processing scale. Journal of Psychosomatic Research, 62(2), 167–178. https://doi.org/10.1016/j.jpsychores.2006.09.005
Brislin, R. (1970). Back-translation for cross-cultural research. Journal of cross-cultural psychology. 1, 185-216.
Cicchetti, D. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological assessment, 6 (4), 284-290.
Cohen, J. (1992). A power primer. Psychological bulletin. 112 (1), 155-159.
Field, A. (2018). Discovering Statistics Using IBM SPSS Statistics—Fifth edition. Sage Publications Inc.
Gay, M.-C., Baker, R., Vrignaud, P., Thomas, P., Heinzlef, O., Haag, P., Banovic, I., & Thomas, S. (2019). Cross-cultural validation of a French version of the Emotional Processing Scale (EPS-25). European Review of Applied Psychology, 69(3), 91–99. https://doi.org/10.1016/j.erap.2019.05.002
Górska, D. & Jasielska, A. (2010). Konceptualizacja przetwarzania emocjonalnego i jego pomiar – badania nad polską wersją skali przetwarzania emocjonalnego Bakera i współpracowników. Studia Psychologiczne, 2, 75-87.
Guadagnoli, E., & Velicer, W. F. (1988). Relation of sample size to the stability of component patterns. Psychological Bulletin, 103(2), 265–275. https://doi.org/10.1037/0033-2909.103.2.265
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
Jasielska, A. & Górska, D. (2013). Badania porównawcze przetwarzania emocjonalnego w poplacjach angielskich i polskicj studentów—Doniesienie z bada´n. Studia Psychologiczne, 4, 5–13. https://doi.org/10.2478/v10167-010-0072-5
Johansson, R., Björklund, M., Hornborg, C., Karlsson, S., Hesser, H., Ljótsson, B., Rousseau, A., Frederick, R. J., & Andersson, G. (2013). Affect-focused psychodynamic psychotherapy for depression and anxiety through the Internet: A randomized controlled trial. PeerJ, 1, e102. https://doi.org/10.7717/peerj.102
Johansson, R., Hesslow, T., Ljótsson, B., Jansson, A., Jonsson, L., Färdig, S., Karlsson, J., Hesser, H., Frederick, R. J., Lilliengren, P., Carlbring, P., & Andersson, G. (2017). Internet-based affect-focused psychodynamic therapy for social anxiety disorder: A randomized controlled trial with 2-year follow-up. Psychotherapy, 54(4), 351–360. https://doi.org/10.1037/pst0000147
Kharamin, S., Shokraeezadeh, A. A., Shirazi, Y. G., & Malekzadeh, M. (2021). Psychometric Properties of Iranian Version of Emotional Processing Scale. Open Access Macedonian Journal of Medical Sciences, 9(B), 217–224. https://doi.org/10.3889/oamjms.2021.5902
Kroenke, K., Spitzer, R. L., Williams, J. B. W., & Löwe, B. (2010). The Patient Health Questionnaire Somatic, Anxiety, and Depressive Symptom Scales: A systematic review. General Hospital Psychiatry, 32(4), 345–359. https://doi.org/10.1016/j.genhosppsych.2010.03.006
Lauriola, M., Donati, M. A., Trentini, C., Tomai, M., Pontone, S., & Baker, R. (2021). The Structure of the Emotional Processing Scale (EPS-25): An Exploratory Structural Equation Modeling Analysis Using Medical and Community Samples. European Journal of Psychological Assessment, 1–10. https://doi.org/10.1027/1015-5759/a000632
Maroti, D., Ek, J., Widlund, R.-M., Schubiner, H., Lumley, M. A., Lilliengren, P., Bileviciute-Ljungar, I., Ljótsson, B., & Johansson, R. (2021). Internet-Administered Emotional Awareness and Expression Therapy for Somatic Symptom Disorder With Centralized Symptoms: A Preliminary Efficacy Trial. Frontiers in Psychiatry, 12, 620359. https://doi.org/10.3389/fpsyt.2021.620359
Orbegozo, U., Matellanes, B., Estévez, A., & Montero, M. (2018). Adaptación al castellano del Emotional Processing Scale-25. Ansiedad y Estrés, 24(1), 24–30. https://doi.org/10.1016/j.anyes.2017.10.006
Petermann, F. (2018). Emotional Processing Scale (EPS-D). Zeitschrift für Psychiatrie, Psychologie und Psychotherapie, 66(4), 256–257. https://doi.org/10.1024/1661-4747/a000364
Rachman, S. (1980). Emotional processing. 18(1), 51-60. https://doi.org/10.1016/0005-7967(80)90069-8.
Sifneos, P. E. (1991). Affect, Emotional Conflict, and Deficit: An Overview. Psychotherapy and Psychosomatics, 56(3), 116–122. https://doi.org/10.1159/000288543
Simonsson‐Sarnecki, M., Lundh, L., Törestad, B., Bagby, R. M., Taylor, G., & Parker, J. (2000). A Swedish Translation of the 20‐item Toronto Alexithymia Scale: Cross‐validation of the Factor Structure. Scandinavian Journal of Psychology, 41(1), 25–30. https://doi.org/10.1111/1467-9450.00167
Spitzer, R. L., Kroenke, K., Williams, J. B. W., & Löwe, B. (2006). A Brief Measure for Assessing Generalized Anxiety Disorder: The GAD-7. Archives of Internal Medicine, 166(10), 1092. https://doi.org/10.1001/archinte.166.10.1092
The jamovi project (2021). jamovi (Version 1.6) [Computer Software]. Retrieved from https://www.jamovi.org

No competing interests reported.

Download PDF

Editorial decision: Major revision
07 Apr, 2022
Reviews received at journal
12 Feb, 2022
Reviewers agreed at journal
12 Feb, 2022
Reviewers agreed at journal
08 Feb, 2022
Reviewers invited by journal
05 Jan, 2022
Editor assigned by journal
05 Jan, 2022
Editor invited by journal
05 Jan, 2022
Submission checks completed at journal
05 Jan, 2022
First submitted to journal
30 Dec, 2021

You are reading this latest preprint version

Psychometric Properties of the Emotional Processing Scale in Individuals with Psychiatric Symptoms and the Development of a Brief 15-Item Version

Status:

Version 1

Abstract

Objective

Method

Results

Conclusions

Figures

Introduction

Internal consistency and dimensionality

Convergent and discriminant validity

Test-retest reliability

Aims of the study with hypotheses

Planned aims

Revised aims

Method

Data

Participants and recruitment

Procedure

Planned analysis

Results

Factor Structure

Confirmatory factor analysis (CFA) of EPS-25

Internal consistency of EPS-25

Convergent and discriminant validity of EPS-25

Test-retest reliability

Exploratory factor analysis (EFA)

Development and dimensionality of the 15-item Emotional processing scale

Statistical considerations

Theoretical considerations

Internal consistency of the EPS-15

Convergent and discriminant validity of EPS-15

Test-retest reliability

Discussion

Inconsistent findings pertaining to the EPS-25

Properties and potential advantages of the EPS-15

Limitations

Overall discussion and future studies

Declarations

References

Additional Declarations

Status:

Version 1