This study demonstrated that the Kidscreen 27 instrument reached good levels of reproducibility, internal consistency, discriminatory power and construct validity, being adequate to measure the HRQoL of adolescents in the Brazilian context.
The reproducibility of this study found higher ICC values ranging from 0.71 to 0.78 than the Ravens-Sieberer at al.[26] study with data from 13 European countries and similar to the studies developed by Andersen et al.[7] (ICC: 0.71 a 0.81) and Nezu et al.[8] (ICC: 0.73 a 0.79). Higher values were found in the studies by Quintero et al.[27] (ICC: 0.87 a 0.99), Ng et al.[28] (ICC: 0.78 a 0.86) and Farias Júnior et al.[13] (0,70 − 0,96). In the present study, the second application of the questionnaire was seven days after the first. Virtually all studies used an interval of seven days or more between applications. Some precautions with the ICC values must be considered, such as the questionnaire application procedure (face-to-face interview, telephone interview, responsible interview (proxy) and self-report) and the age range of the respondents[29]. Specifically regarding administration, Kidscreen was originally developed to be responded through self-reported or parents/guardians[1]. The administration by interview was observed in the study by Farias Júnior et al.[13] and Quintero et al.[27], a question that may explain the slightly higher ICC values, in relation to the values found in this study.
A moderate to good agreement was observed by the Gwet coefficient between the items of the five dimensions, with most items classified as “good agreement” according to the criteria by Landis & Koch[20]. Item 1 “In general, how would you say your health is?”, Was the item with the highest agreement (0.83), followed by item 25 “Have you got on well at school?” (0.81). The other items-maintained values ranging from 0.73 to 0.60. On this issue, the literature has highlighted the frequent use of the Kappa coefficient instead of the Gwet coefficient[30] to analyze the stability of categorical variables. Although Kilen Li Gwet proved in 2002[19] the superiority of the Gwet coefficient when compared to Cohen's Kappa, few researchers use it as a statistical tool, or are not even aware of its existence[30].
The internal consistency of this study was assessed by McDonald's omega coefficient, an estimator that considers the different standardized factor loads for each item of the instrument. The omega values in the present study ranged from 0.82 for the School Environment dimension to 0.91 for the Psychological Well-being dimension, classified as desirable[21].
The decision by this estimator contradicts most of the validation studies for Kidscreen, which chose to present Cronbach's alpha[31], including by the Kidscreen Group[1]. In view of this, it was decided to present the alpha values as well, as no psychometric study of the Kidscreen instrument was found in the literature involving the omega coefficient for internal consistency analysis. Cronbach's alpha values in the present study ranged from 0.76 for the School environment dimension to 0.82 for the Physical Well-being dimension, values similar to those found by the Kidscreen Group (0.80–0.84)[1], Andersen et al. (0.77 a 0.82)[7] and Shannon et al. (0.65–0.74)[9]. Following Cronbach’s criteria[31], it can be said that the scale has acceptable internal consistency.
The scale's discriminatory power ranged from 0.94 to 0.98. These results are similar to found by the Kidscreen Group (0.81–0.99). This analysis, even recommended[1], has not yet been reported by any study analyzing the psychometric properties of Kidscreen. In this study, we opted for the Hankins Delta G coefficient[24], as it is theoretically more appropriate for the type of items on the Kidscreen, polytomous and graduated response scale.
The CFA supported the five dimensions found in the original study and in other studies[7,13], demonstrating that the instrument is in accordance with the conceptual and theoretical considerations on the measurement of HRQoL. The loads of the items as well as the correlations between the dimensions were considered good and the multidimensional structure was confirmed. The final model showed an acceptable fit, especially when the modification indices were taken into account. The estimator WLSMV used in this analysis, uses the matrix of polychoric correlations between the items during the factor analysis. Correlations of this nature tend to be, in comparison to Pearson's coefficient, a more consistent estimate of the true linear relationship between variables[23]. Kidscreen validity studies use maximum likelihood as an estimator for exploratory and confirmatory factor analyzes[9,13].
In the model, three covariance were added, in this order: (i18-i19; i9-i10; i10-i11). Conceptually, the covariance of these items makes sense. Item 18 “Have you had enough money to do the same things as your friends?” and item 19 "Have you had enough money for your expenses? ", both of the dimension "Autonomy and Parent relation" are related to financial issues, unlike the other items (13, 14, 15, 16 and 17) that are related to issues of autonomy, relationship with parents and life at home. Although parents are probably at this age, sources of financial support, this seems to be independent of their relationships with them[28].
Other covariance, from item 9 “Have you felt sad?” with item 10 “Have you felt so bad that you didn’t want to do anything?” and item 10 “Have you felt so bad that you didn’t want to do anything?” with item 11 “Have you felt lonely?”, are related to mood and emotion, different from the other items (6, 7, 8 and 12) that are related to psychological characteristics (items 6, 7 and 8) and self-perception (item 12). Andersen et al. (2016)[7], when evaluating the psychometric properties of the Norwegian version of Kidscreen 27, draw attention to another detail: items 9, 10 and 11 are formulated “negatively”, different from the other items that are written “positively”, which can influence the contribution of the items to the dimension in question.
In contrast to these findings, two studies confirmed other dimensional structures for Kidscreen in its version with 27 items. Ng et al.(2015)[28] evaluated the original proposed five-dimensional model, but the fit was poor: X2 [df] = 4,553.16 [314], p < 0.001; RMSEA = 0.10; CFI = 0.91; TLI = 0.90). After employing the resources of the TRI, through Rasch modeling a seven-dimensional model was identified, presenting a new structure with acceptable fit (X2 [df] = 2,507.43 [303], p < 0.001; RMSEA = 0.07; CFI = 0.95; TLI = 0.95). A study conducted by Quintero et al. (2011)[27] performed exploratory factor analysis and confirmed a seven-dimensional version. When excluding item 1 “In general, how would you say your health is?”, Six dimensions remained. At the end, the confirmatory factorial ratified the six dimensions (RMSEA = 0.097; CFI = 0.754; NFI = 0.699; GFI 0.754; AGFI = 0.701).
This study had strengths. Theoretical development and validation of instruments for measuring quality of life in adolescents has become relevant in different contexts in the health field, as it is a recognized way to understand the needs in health services and guide decision making for the allocation of financial resources for health programs[32]. In addition, methodological rigor of processing and analysis of variables was followed as recommended by the Kidscreen Group, which allows external comparisons.
Limitations included the fact that sensitivity to change was not tested due to the objectives and cross-sectional design of this study. An important fact to be highlighted is that in the version of the Kidscreen 27 instrument, the Self-Perception dimension is represented only by item 12 "Have you been happy with the way you are?". Originally, the construction of the instrument started from a generic proposal, indicated to measure the HRQoL of healthy children and adolescents and/or with chronic diseases. The sample of this study did not count on the participation of adolescents who had physical limitations, which could be interesting to test the psychometric properties.