The primary research aim was to investigate the psychometric properties of the BYSAS. Face validity, content validity, exploratory factor analysis (in the first sample), and confirmatory factor analysis (in the second sample) were used to assess the validity and Cronbach’s alpha coefficient and test-retest reliability.
Validity
Face validity: Face validity is the objective judgment of the instrument and responds from the point of view of the target group that is the designed tool seemingly relevant to the purpose of the study? Do people who want to respond to the tool, agree with the tool's wording and expressions? Is the perception of non-specialists (the target group) the same as the researcher intended? Are the components and totality of the instrument acceptable to respondents? In this study, qualitative and quantitative methods were used to determine face validity of the instrument. In the qualitative method and at the beginning of the face validity determination process, a team of psychologists and psychometrists with relevant research backgrounds were asked to evaluate the tool questions to determine the appropriateness of the words and sentences for the target group. In some cases, items were changed to make them easier and more understandable.
To get the target group's comments, an interview was also conducted with a sample of participants to identify difficulty in understanding the wording and phrasing of items, the appropriateness and relevance of the items, the likelihood of ambiguity and inaccurate interpretations of the phrases, or the inaccuracy of word meanings and their comments were made as minor changes to the questionnaire. The face validity of the measures was quantitatively by the item impact method. For this purpose, a 5-point Likert scale was considered for each item of the measure: extremely important (5 points), important (4 points), moderately important (3 points), slightly important (2 points) and not at all important (1 points). Then, the questionnaire was administered to 20 individuals of the group to determine face validity and after completing the questionnaires, face validity was calculated using an item impact formula. To accept the face validity of each item, its impact score should not be less than 1.5 and only face validity questions are acceptable which is a score above 1.5. As Table 2 shows, all items reported acceptable face validity.
Content validity: Content validity answers questions such as: does the designed tool include all the important aspects of the concept of measurement? Does the tool item measure what it should? In this study, both qualitative
and quantitative methods were used to determine content validity. A qualitative method was consulted with clinical psychology and psychometrics specialists about quality and adequacy of items. After collecting the expert evaluation, the required changes to the tool were considered and applied in consultation with the members of the research team. Content validity was calculated quantitatively based on the experts' opinions and by calculating both content validity ratio (CVR) and content validity index (CVI).
The content validity ratio was used to ensure that the most important and correct content (item necessity) was selected and the content validity index was used to ensure that the tool items were best designed to measure content. The content validity ratio quantitatively was used by 10 experts in clinical psychology and psychometrics who were all faculty members of Iranian universities in order to respond to each of the tool items or metrics used in the three item ranges (including essential, useful, but not necessary, and not necessary).
The referee’s opinion was calculated as follows:
The formula for the content validity ratio is the number of evaluators who consider the item necessary or useful and N is the total number of evaluators or reviewers who have reviewed the item.
The CVI index was calculated after determining the CVR. To calculate this index, evaluators commented on each item of the instrument used (the Bergen–Yale Sex Addiction Scale), based on the four criteria of relevance or propriety, simplicity and fluency and clarity or transparency, based on a 4-point Likert scale. For example, the criterion was used for the relevance of options (not relevant = 1, relatively relevant = 2, and relevant = 3 and fully relevant = 4). Then, the content validity index was calculated using the CVI formula.
ne3, 4: The number of ratings given the score of 3 and 4.
N: Total number of evaluators.
The minimum score required for acceptance of content validity ratio (CVR) according to the Laosche method for 10 expert members was 0.80 and the minimum content validity index (CVI) was 0.75. According to the results, all items on the scale had good validity.
Exploratory Factor Analysis: To determine the number of factors, the Kaiser criterion was used which retains factors with eigenvalues greater than 1. Only one factor has this criterion. Principal component was used for data analysis and varimax rotation was used for extraction of factors.
Data from 376 participants (34% male and 66% female) was used for this phase. Only 8 questionnaires were incomplete. Four questionnaires were excluded from the study due to participant's withdrawal from the study in order to adhere to ethical principles and avoid selective bias. Data on 364 questionnaires were analyzed. The eigenvalue was obtained for 3.74, which explains the variance of 62.31. No questions were deleted and the load factor of the remaining questions is between 0.76 and 0.84. The results are reported in Table 4.
As Table 4 shows, all factor loadings on items were higher than 0.7, indicating that the questions are significantly related to their underlying factor. As mentioned above, with respect to the sample size, the factor loadings above 0.40 are significant. The bottom column of the table above shows the subscription rate for each of the questions. This value represents the amount of variance explained by each factor extracted. For example, 69% of the variance of the first question is explained by the underlying factor. Below the table is reported the specific amount and variance explained by the underlying factor. The eigenvalue represents some of the total variance of the variables by which the agent is explained. The total amount of variance explained in the model is 62.31%, reflecting a unidimensional factor measuring sex addiction.
Confirmatory factor analysis: A confirmatory factor analysis was conducted to confirm the factor structure reported in the first sample. If the structure obtained from the first sample to the second sample is confirmed, a certificate is provided for its validity. Lisrel8.8 software was used for confirmatory factor analysis. Weighted Least Squares estimation method was used in data analysis with data from polychoric correlation and asymptotic covariance matrix. The Weighted Least Squares method was preferred because the query options were four-class and Polyureic correlation was calculated instead of Pearson correlation (12). There are two types of evaluations to consider in confirmatory models. Partial evaluation and overall fit of the model. The Partial evaluation relates to paths drawn from current agents to the markers (in measurement model). The overall fit of the measurement models was judged using several goodness of fit indices (which measure the amount of data support for the conceptual model). The indices used are: Chi-square exponential ratio (X2), Chi-square to degrees of freedom (X2 / df), goodness of fit (GFI), modified goodness of fit (AGFI), root mean square error of estimation (RMSEA), CFI and NFI and the Tucker-Lewis Index (TLI).
Figure 1. shows the results of the confirmatory factor analysis and the fit indices. These indicators provide the information to evaluate the overall fit of the model. The fit indices support the proposed one-factor model.
According to the information in this table, out of the eight indices reviewed, six indices are in favorable condition and only NFI and RMSEA are in relatively favorable condition. Based on the results, it can be said that the overall fit of the measurement model is in the optimum condition.
Reliability: Cronbach's alpha coefficient of internal consistency for the instrument was 0.88 (95% confidence intervals: 0.83-0.83), suggesting the scale has good internal consistency. The mean inter-item correlation was 0.54. In addition, the intra-class correlation coefficient was obtained between the two test runs with two weeks intervals for 23 persons equal to 0.89, which is an acceptable value (p-value=0.009). Therefore, it shows that the reliability of sex addiction test is valid.