Forward Translation, Expert Panel Discussion, and Back Translation
After the IAT questionnaire was translated into Indonesian, it was reviewed by three experts in a panel discussion. The terms “online” and “offline” that had been preserved in the English language by the translators were replaced with their parallel Indonesian counterparts. The term “depression,” however, was retained since it is a considerably well-known term among teenagers. The revised questionnaire from the expert panel discussion was translated back into English. The results of the backward translation were then shared with the original questionnaire creator, Dr. Kimberly Young from Net Addiction, the Center of Internet Addiction and approval was obtained.
Face Validity Study
A face validity study was conducted to gather the construct validity and reliability of the IAT questionnaire through a focus group discussion (FGD). A total of 30 students (15 JHS students and 15 SHS) participated in the study. Nine females and 21 males took part in the study with ages ranging from 12–18 years. The characteristics of the subjects are listed in
Table 1.
All participants were asked to fill in the Indonesian version of the IAT that consisted of 20 items before the FGD. During the FGD, the students made some suggestions pertaining to the terms used in the questionnaire to make them more familiar to teenagers. Changes were made in several statements without altering their intended meaning. The students also did not know about the term “log in” because currently, electronic devices do not require to be logged into. The term “log in” was therefore changed to “online.” The term “online” was also considered for replacement with the term “playing internet” since being online does not necessarily indicate active internet use. The word “couple” is also not suitable for teenagers; therefore, it was replaced with “family,” “friends,” or “closest person.” The term “pasangan” (lit. couple) is also considered to imply a romantic relationship and therefore was replaced with “orang - orang terdekat” (lit. relatives). “Work productivity” is also not applicable to teenagers and was replaced with “academic achievement.”
The Pearson correlation test was carried out between each item with the total score to assess the validity of the IAT questionnaire. It was observed that all items were valid since the correlation between items was above 0.3 (ranging from 0.419 to 0.788). The questionnaire also exhibited very good reliability with an α-Cronbach value of 0.913.
Following the face validity study, the results were discussed by the three experts. The altered terms are listed in Table 2. Next, a consultation was held with Indonesian language experts from the Faculty of Literature, University of Indonesia. Minor modifications were made in some sentences in accordance with adolescents’ understanding level and in order to emphasise the idea better.
Pilot Study
In the pilot study, the assessment of IAT validity and reliability was conducted on 385 subjects (145 JHS and 240 HS students). The subjects in the field test were different from those involved in the focus group discussion. The ratio of male and female participants was 47.5%:52.5%. The field test revealed that most participants used the internet anywhere and by using a modem (54.8%). The majority surfed the internet for 4–8 hours per day (61%); however, 5.7% of the participants used the internet more than 8 hours per day. The aim of using the internet was mostly to play online games (81.8%). From 20 items in the IAT questionnaire, the corrected item-total correction test was conducted. It was revealed that the values for item-total correlations ranged from (0.206–0.577). Item number 7—“How often do you check your email before doing the other activities that you need to do?”—was found to have a correlation value of 0.206, thereby indicating poor reliability. Hence, the item was deleted. The internal correlation was retested and the result showed a good correlation (above 0.3) for each of the 19 questionnaire items. Values for the item-total correlations ranged from 0.316 to 0.576. Moreover, the internal reliability coefficient was 0.862.
Psychometric Evaluation Study
Following the pilot study, a total of 643 subjects (333 JHS students and 310 SHS students) participated in the field test for the psychometric evaluation study. The characteristics of the participants are described in Table 1.
An EFA was next performed in the 19-item IAT questionnaire. From the first EFA, item number 5—“How often do people in your life complain about the amount of time that you spend to play on the Internet?”—was found to have loading factors < 0.4, thereby indicating poor validity. This item was hence deleted.
Next, the second EFA for the 18-item IAT was conducted. It showed four factors or domains with eigenvalues more than one and explained 52.557% of the total variance. The grouping of the factors was based on the highest loading factor within the particular domain with a minimal value of the loading factor equal to or more than 0.4. The results showed that each item has a satisfactory loading factor (> 0.4). However, domain 3 consisted of only two items and did not fulfill the minimum requirement of three items. Consequently, we performed a re-run of the analysis and specifically asked for three components.
Subsequently, the third EFA was run. Unlike the first and second EFA, eigenvalue was not used to determine the domain in the third EFA since the domains had been decided from the beginning by the determining extract factor. The third EFA revealed three domains and all items had a loading factor > 0.4. The factor loads related to the 18 items ranged from 0.449 to 0.850, thereby indicating that these questions were sufficiently qualified to be included in the test. The three domains, along with the factor loadings, are listed in Table 2. The three domains were named salience, neglect of duty, and loss of control.
Cronbach’s alpha coefficient was also calculated to measure the reliability of the instrument. The reliability of model 2, which consisted of three domains and 18 items, was analysed. Values for item-total correlations ranged from 0.317 to 0.574 with the internal consistency value of the Cronbach alpha coefficient being 0.855. The values of the reliability coefficient for each domain are listed in Table 3.
Two models were assessed in this study: the first model used the original version of IAT (one domain, 20 items) while the second model is in accordance with the EFA results (three domains, 18 items).
CFA’s first model resulted χ2 (df = 152, p < 0.001) = 488.05 and χ2 /df = 3.21 with RMSEA = 0.059, CFI = 0.97, SRMR = 0.046, and AIC = 604.05. While the second model generated χ2 (df = 148, p < 0.001) = 700.63 and χ2 /df = 4.73 with RMSEA = 0.076, CFI = 0.95, SRMR = 0.057, and AIC = 784.63 (Table 4). The results of each model were subsequently compared.
The first parameter that should be evaluated is the Akaike Information Criterion (AIC). The more efficient model has a lower AIC score [18] and, in this case, was the second model. Following this, other parameters were also compared and it was discovered that the second model exhibited a higher value than the first model in all goodness of fit indices. Hence, the second model was the preferred model in this study. The results of the CFA of the second model are given in Figure 1.
This study also analysed the relationship between the extent of internet addiction, age, gender, and onset, duration, and aim of internet use. The level of internet addiction was determined through IAT scores (normal, mild, moderate, and severe addiction) [7]. The cut-off scores for categorising internet addiction were formulated in the 18-item Indonesian version of the IAT since it exhibited better psychometric properties than the 20-item one. We used the same method adopted by Karim et al. in Bangladesh (2014) [19] as a reference to categorise the Indonesian version of the IAT (Table 5).
For our data analysis, the internet addiction level was simplified into two categories: normal and mild addiction subjects grouped together and moderate internet addiction group. A severe addiction group was not included since none of the subjects exhibit severe addiction as per our study.
A significant association was revealed between the extent of internet addiction and gender (x2(df) = 4.921(1), p = 0.027, OR = 1.669, CI = 1.081–2.577) and duration of internet use per week (x2(df) = 5.094(1), p = 0.024, OR = 0.545, CI = 0.329–0.905). Meanwhile, no significant association was observed between the extent of internet addiction and age, aim, and onset of internet use (p > 0.05).