The primary research aim was to investigate the psychometric properties of the BYSAS. Face validity, content validity, exploratory factor analysis (in the first sample), and confirmatory factor analysis (in the second sample) were used to assess the validity and Cronbach’s alpha coefficient and test-retest reliability.
Validity
Face validity
Face validity is the objective judgment of the instrument and responds from the point of view of the target group that is the designed tool seemingly relevant to the purpose of the study? Do people who want to respond to the tool, agree with the tool's wording and expressions? Is the perception of non-specialists (the target group) the same as the researcher intended? Are the components and totality of the instrument acceptable to respondents? In this study, qualitative and quantitative methods were used to determine face validity of the instrument. In the qualitative method and at the beginning of the face validity determination process, a team of psychologists and psychometrists with relevant research backgrounds were asked to evaluate the tool questions to determine the appropriateness of the words and sentences for the target group. In some cases, the item use has been changed to make it easier and more understandable.
To get the target group's comments, an interview was also conducted with a sample of experts to identify difficulty in understanding the wording and phrasing of items, the appropriateness and relevance of the items, the likelihood of ambiguity and inaccurate interpretations of the phrases, or the inaccuracy of word meanings and their comments were made as minor changes to the questionnaire. The face validity of the measures was quantitatively by the item impact method. For this purpose, 5-point Likert scale was considered for each item of measure: extremely important (5 points), important (4 points), moderately important (3 points), slightly important (2 points) and not at all important (1 points). Then, the questionnaire was administered to 20 individuals of the group to determine face validity and after completing the questionnaires, face validity was calculated using item impact formula. To accept the face validity of each item, its impact score should not be less than 1.5 and only face validity questions are acceptable which is score above 1.5. As Table 2 shows, all items reported acceptable face validity.
Table 2
Result of the face validity
Item no. | Effect size | ne/N | Mean | not at all important (1 points) | slightly important (2 points) | moderately important (3 points) | important (4 points) | Extremely important (5 points) |
1. Spent a lot of time thinking about sex/masturbation or planned sex? | 2.96 | 0.75 | 3.95 | 0 | 1 | 3 | 7 | 8 |
2. Felt an urge to masturbate/have sex more and more? | 3.61 | 0.85 | 4.25 | 0 | 1 | 2 | 8 | 9 |
3. Used sex/masturbation in order to forget about/escape from personal problems? | 3.24 | 0.80 | 4.05 | 1 | 1 | 2 | 8 | 8 |
4. Tried to cut down on sex/masturbation without success? | 3.28 | 0.80 | 4.10 | 3 | 2 | 2 | 8 | 8 |
5. Become restless or troubled if you have been prohibited from sex/masturbation? | 3.28 | 0.80 | 4.10 | 0 | 2 | 2 | 8 | 8 |
6. Had so much sex that it has had a negative impact on your private relationships, economy, health or job, studies? | 3.82 | 0.90 | 4.25 | 0 | 2 | 0 | 9 | 9 |
Table 3
Result of the content validity
Item no. | Ne | CVI | CVR |
1 | 10 | 916.0 | 1 |
2 | 10 | 966.0 | 1 |
3 | 9 | 933.0 | 8.0 |
4 | 10 | 941/0 | 1 |
5 | 10 | 9330. | 1 |
6 | 9 | 908.0 | 0.8 |
Content validity: Content validity answers questions such as: does the designed tool include all the important aspects of the concept of measurement? Does the tool item measure what it should? In this study, both qualitative and quantitative methods were used to determine content validity. Qualitative method was consulted with clinical psychology and psychometrics specialists about quality and adequacy of items. After collecting the expert evaluation, the required changes to the tool were considered and applied in consultation with the members of the research team. Content validity was calculated quantitatively based on the expert's opinions and by calculating both content validity ratio (CVR) and content validity index (CVI).
The content validity ratio was used to ensure that the most important and correct content (item necessity) was selected and the content validity index was used to ensure that the tool items were best designed to measure content. The content validity ratio quantitatively was used by 10 experts in clinical psychology and psychometrics who were all faculty members of Iranian universities in order to respond to each of the tool items or metrics used in the three item ranges (including essential, useful, but not necessary, and not necessary).
The referee’s opinion was calculated as follows: (see Formula 1 in the Supplementary Files)
The formula for the content validity ratio is the number of evaluators who consider the item necessary or useful and N is the total number of evaluators or reviewers who have reviewed the item.
The CVI index was calculated after determining the CVR. To calculate this index, evaluators commented on each item of the instrument used (Dual Dangerous Driving Index), based on the four criteria of relevance or propriety, simplicity and fluency and clarity or transparency, based on a 4-point Likert scale. For example, the criterion was used for the relevance of options (not relevant = 1, relatively relevant = 2, and relevant = 3 and fully relevant = 4). Then, the content validity index was calculated using the CVI formula (see Formula 2 in the Supplementary Files).
ne3, 4: The number of ratings given the score of 3 and 4.
N: Total number of evaluators.
The minimum score required for acceptance of content validity ratio (CVR) according to the Laosche method for 10 expert members was 0.62 and the minimum content validity index (CVI) was 0.75. According to the results, all items on the scale had good validity.
Exploratory Factor Analysis
Data from 376 participants (34% male and 66% female) was used for this phase. Only 8 questionnaires were incomplete. Four questionnaires were excluded from the study due to participant's withdrawal from the study in order to adhere to ethical principles and avoid selective bias. Data on 364 questionnaires were analyzed. The results are reported in Table 4.
Table 4
One-factor confirmatory factor analysis results for the BYSAS
Item no. | Subscription rate | factor loadings |
1 | 0.71 | 0.83 |
2 | 0.64 | 0.80 |
3 | 0.63 | 0.77 |
4 | 0.69 | 0.84 |
5 | 0.59 | 0.80 |
6 | 0.52 | 0.76 |
% of Variance: 62.31, special value: 3.74 |
As Table 4 shows, all factor loadings on items were higher than 0.7, indicating that the questions are significantly related to their underlying factor. As mentioned above, with respect to the sample size, the factor loadings above 0.40 are significant. The bottom column of the table above shows the subscription rate for each of the questions. This value represents the amount of variance explained by each factor extracted. For example, 69% of the variance of the first question is explained by the underlying factor. Below the table is reported the specific amount and variance explained by the underlying factor. The eigenvalue represents some of the total variance of the variables by which the agent is explained. The total amount of variance explained in the model is 62.31%, reflecting a unidimensional factor measuring sex addiction.
Confirmatory factor analysis: A confirmatory factor analysis was conducted to confirm the factor structure reported in the first sample. If the structure obtained from the first sample to the second sample is confirmed, a certificate is provided for its validity. Lisrel8.8 software was used for confirmatory factor analysis. Weighted Least Squares estimation method was used in data analysis with data from polyureic correlation and asymptotic covariance matrix. The Weighted Least Squares method was preferred because the query options were four-class and Polyureic correlation was calculated instead of Pearson correlation (12). There are two types of evaluations to consider in confirmatory models. Partial evaluation and overall fit of the model. The Partial evaluation relates to paths drawn from current agents to the markers (in measurement model). The overall fit of the measurement models was judged using several goodness of fit indices (which measure the amount of data support for the conceptual model). The indices used are: Chi-square exponential ratio (X2), Chi-square to degrees of freedom (X2 / df), goodness of fit (GFI), modified goodness of fit (AGFI), root mean square error of estimation (RMSEA), CFI and NFI and the Tucker-Lewis Index (TLI).
Figure 1. shows the results of the confirmatory factor analysis and the fit indices. These indicators provide the information to evaluate the overall fit of the model. The fit indices support the proposed one-factor model.
According to the information in this table, out of the eight indices reviewed, seven indices are in favorable condition and only NFI is in relatively favorable condition. Based on the results, it can be said that the overall fit of the measurement model is in the optimum condition.
Reliability
Cronbach's alpha coefficient of internal consistency for the instrument was 0.88, suggesting the scale has good internal consistency. The mean inter-item correlation was 0.54. In addition, the intra-class correlation coefficient was obtained between the two test runs with two weeks intervals for 23 persons equal to 0.89, which is an acceptable value (p-value = 0.009). Therefore, it can be concluded that the reliability of sex addiction test is valid.