Emotion regulation (ER) comprises a set of strategies (cognitive, emotional and physiological) that allow the individual to face various internal or external stimuli, managing the emotional response in order to adapt to their environment and achieve goals (Gross, 1999; Gross & John, 2003). Research in ER has grown exponentially due to the important role it plays in social adaptation and in the development of certain psychopathologies (Aldao, Nolen-Hoeksema & Schweizer, 2010), but also in the integral development of a human being (Momeñe, Jauregui & Estevez, 2017).
The Emotion Regulation Questionnaire (ERQ) (Gross & John, 2003) is used to evaluate ER, that assesses two independent regulation strategies: a) Cognitive Reappraisal (CR), which is an anticipatory strategy that allows interpretation and evaluation of context before the emotional response in order to modulate behavior when faced with a triggering stimuli; and b) Emotion suppression (ES) which allows the modulation of emotions while the individual experience them (Gross, 2007; Aldao et al., 2010; Balzarotti et al., 2010).
The ERQ has several translations and validations around the world, some of them present evidence of a two-factor orthogonal model (CR and ES without correlation) in Italy (Balzarotti et al., 2010); Germany (Abler & Kessler, 2009); Spain (Rodríguez-Carvajal et al., 2006; Cabello et al., 2013); Portugal (Teixeira et al., 2015); Australia and United Kingdom (Spaapen et al., 2014) And USA (Preece et al., 2021); while others show evidence of a two-factor oblique model (CR and ES correlated) in Sweden (Enebrink et al., 2013); Peru (Gargurevich & Matos, 2010); Ecuador (Moreta-Herrera et al., 2018) and Australia (Preece et al., 2020). Both two-factor adjustment models (orthogonal and oblique), present an adequate internal consistency reliability as well as convergent validity when compared with other tests (health, well-being, emotional intelligence, among others).
Despite the promising results presented in these studies, several methodological errors have been found, such as in the selection of specific statistical tests that strictly comply with estimators for criterion and factorial validity as well as with internal consistency, which are quite important in the process for adaptation and validation of a test. Therefore, these errors can bias the conclusions. Thus, it is necessary to carry out new confirmatory studies, which include adequate methodological corrections taking into account the nature of data, in order to guarantee the validity of the ERQ for subsequent proper use.
Methodological implications on the validation of tests
Having tests translated, adjusted and adapted to the context in which the ERQ is applied, as well as any other test, is one of the challenges of evidence-based instrumental research. Nowadays, empirical research has focused more on social and psychic phenomena rather than on the development and validation of assessment tools. Latin America, including Ecuador, for example, is a region where advances in psychometrics and instrumental studies are scarce and limited. The use of assessment tools without proper instrumental validation, can compromise results from the beginning, due to the absence of calibration (Moreta-Herrera et al., 2019), which leads to measurement errors and biases (Elosua, 2003). This can also cause errors in decision making, testing hypothesis and diagnosis (Rönkkö et al., 2015).
An inadequate interpretation of the response to an item of a test (commonly answered with a Likert-type scale) is an error that results in the misuse of statistical tests during validation processes. Regardless of the number of options, their nature tends to be more categorical or ordinal rather than continuous, however, this type of data ends up receiving analysis through tests that assume normality and continuous nature, when in fact it is not (Sullivan & Artino, 2013). This error is observed in different statistical validation processes such as exploratory factor analysis (EFA), confirmatory factor analysis (CFA), construct validity and reliability, to name a few examples.
Considerations in Confirmatory Factor Analysis and Reliability
CFA is a statistical method widely used as evidence for the construct validity of a measure (Ferrando & Anguiano-Carrasco, 2010). This requires a considerable sample size (Brown, 2015), the confirmation of multivariate normality (Cain et al., 2017) and the type variables (categorical, ordinal or interval) (Hair et al., 2004). The treatment of data will depend on whether or not these criteria are met, using normal or robust estimators.
CFA is generally calculated with Maximum Likelihood Estimation method (ML) (Li, 2016), which assumes that the observed indicators (items) follow a continuous and multivariate normal distribution (Myung, 2003). In the case of psychological tests, this is not the most suitable method, as items usually have an ordinal nature (Gitta & Bengt, 2009) and continuous multivariate normal distribution is unlikely (Holtmann et al., 2016). Therefore, CFA requires suitable estimators to these characteristics such as Diagonally Weighted Least Squares (DWLS) method or robust estimations such as Robust Maximum Likelihood (MLR) or Weighted Least Squares with Adjusted Mean and Variance (WLSMV) (Jin & Cao, 2018). These methods, especially MLR are recommended, as it reduces biases compared to ML. This helps to obtain stronger evidence of validity, regardless the number of categories of the item and without multivariate normal distribution as long as large sample size is analyzed (n> 200) (Li, 2016).
Previous studies confirm an orthogonal two-factor model (Rodríguez-Carvajal et al., 2006; Abler & Kessler, 2009; Balzarotti et al., 2010; Cabello et al., 2013; Spaapen et al., 2014; Teixeira et al., 2015; Preece et al., 2021) of the ERQ (Gross & Jhon, 2003); although an alternative configuration of an oblique two-factor model is also proposed (Gargurevich & Matos, 2010; Enebrink et al., 2013; Moreta-Herrera et al., 2018; Preece et al., 2020). The different configurations of the models in these studies are probably due to particular characteristics of the reference samples, differences in language and the estimators used in factor analysis (ML estimation is predominant in validation studies, which induces a greater measurement bias) (Jonason et al., 2020; Moreta-Herrera et al., 2020; Caycho-Rodríguez et al., 2021).
In addition, due to the presence of moderate factor correlations in preliminary studies, it is likely that there is a third latent factor that groups all the items of the scale into a single factor, this would be explained through a bifactor model composed of a general factor (GF) and two specific factors (SF). This model represents the multidimensionality of the construct, and recognizes the uniqueness of the factors that compose it (Stefansson et al., 2016).
Something similar occurs when determining reliability based on internal consistence of the ERQ, which is verified through Cronbach's alpha coefficient (α) (Sijtsma, 2009), a test that requires a significant number of cases for its analysis, as well as a continuous multivariate normal distribution. However, evidence suggests that using Cronbach's alpha is not ideal for this purpose (Trizano-Hermosilla & Alvarado, 2016), due to the ordinal nature of the items and Cronbach's alpha does not consider this aspect. In fact, its use is recommended only when the measurement scale has six or more options and normal distribution assumption is met (Elosua Oliden & Zumbo, 2008). As a result, researchers run the of underestimate or overestimate the true reliability of the measure, therefore, its use is not recommended (Ventura-León & Caycho-Rodríguez, 2017). Given this situation, it is methodologically correct to use reliability estimators according to the nature of the items, such as the omega coefficient (McDonald, 1999), which shows less bias in the assessment of reliability (Dunn et al., 2014) or the ordinal coefficient alpha (Elosua Oliden & Zumbo, 2008).
Given these antecedents, there are still doubts that still need to be clarified about the best factorial fit of the ERQ, as well as other psychometric properties such as reliability, for their correct use in social research and intervention. This, especially in the Latin American and Spanish-speaking population.
Objectives and hypotheses
Based on the analysis contained in this text, the objectives of this study are a) Identify the best fit model of the ERQ (see figure 1) using Robust Maximum Likelihood estimation (MLR) in a sample of Ecuadorian college students, considering an orthogonal and oblique two-factor models as well as a bifactor model with a general factor. It is hypothesized that the bifactor model is the model that best represents the ERQ; b) Estimate the reliability based on internal consistency of the ERQ model with the best fit. It is considered that the ERQ has an optimal and adequate adjustment for the Ecuadorian college students.