Study setting and design
This research was conducted during a cross-sectional, multi-center survey of the qualitative study. Data collection started in April 2024 to June 2024, and all participants completed informed consent.
The study was carried out in four phases. In the first phase, the Chinese Insulin Medication Literacy Scale (Ch-InMLS) for Patients with Diabetes Mellitus pool items were developed. In the second phase, content validity was conducted by twelve experts working in diabetes mellitus management or care, they were invited for a two-round Delphi. In the third phase, 200 patients with diabetes mellitus were tested in the pilot study, the items were revised or deleted by item discrimination analysis, correlation coefficient method and homogeneity test. In the fourth phase, formal investigation was conducted among 553 patients with diabetes mellitus by construct validity and reliability analysis. Figure 1 shows the process of questionnaire development.
Participants
Participants were recruited from the First Affiliated Hospital of Zhengzhou University, which is a Grade-A tertiary general hospital tertiary hospital and two community health service centers in Zhengzhou city of China. Inclusion criteria were as follows: (1) diagnosed with type 1 or type 2 diabetes according to the 1999 WHO diagnostic criteria for diabetes, (2) aged over 18 years, (3) have the ability to read and write or use WeChat, (4) have been on insulin treatment for at least four weeks, included both those who were newly diagnosed and being treated with insulin for a short period of time and those who were already on insulin treatment (once or more in a day) for a longer period of time. The exclusion criteria included: (1) patients with a history of cognitive impairment or psychiatric disease, (2) patients who declined participation, (3) patients with hearing and communication disability.
Phase 1: Development of Ch-InMLS
The development of the questionnaire consisted of the following steps: item generation, semi-structured interviews, cognitive interviews.
Item generation
For the present study, we conducted a comprehensive literature review, existing instruments not only for assessing medication /pharmacotherapy/ pharmaceutical literacy level of patients with diabetes mellitus, but also in the field of adherence/ psychological resistance/ self-management/ attitude about insulin treatment, were searched and studied in PubMed, Web of Science, Embase, Wan Fang and CNKI Data. Studies including reviews and original articles on the definition, fields and dimensions of medication literacy were also included.
The deductive method was conducted to generate items[32], which involved using current literature and insulin-related guidelines to generate items related to each domain (knowledge, attitude, practice and skill). Besides, we referred to guidelines related to insulin to enrich the connotation and composition of Ch-InMLS. Soon afterwards, an initial draft was developed in English, subsequently translated to Chinese using the back-translation method.
Semi-structured interviews
In the stage, we conducted face-to-face individual interviews on experts and patients with diabetes mellitus. Questions asked towards five endocrinologists were "What do you think are significant in insulin self-management in clinical practice?” and “What misunderstandings do you find in insulin therapy for diabetics?”. Questions asked towards eight patients were "Do you have any strengths and barriers with insulin use and what are they?". Items extracted from the review data were developed based on the four core elements of insulin literacy for diabetes mellitus patients of knowledge, attitude, skill, and practice.
In this stage, inductive method was used, combined with the above deductive method, initial drafted scale was developed.
Cognitive interviews
Cognitive interviews and pre-test of the initial drafted scale were conducted in 10 patients with diabetes mellitus through face-to-face individual interviews. The patients were invited to check the readability, comprehensibility, and response errors of the draft scale. Feedback and advice as well as questions put forward by interviewed patients on each item were recorded, and complex items with technical words which were hard to understand were replaced by more popular terms. Then, the researchers communicated with the participants and formulated an original scale based on the participants’ feedback and advice.
Phase 2: Content validity
A panel of twelve experts have been invited to appraise on the construct and items of the primary insulin literacy scale with 37 items in this study. The inclusion criteria of the expert panel: (1) with over 8 years practical experiences of diabetes mellitus management or care, (2) have intermediate or senior titles, (3) experts specializing in the development and psychometric validation of a scale, (4) are willing to participate in our study. Experts assessed expression, grammar, phrasing, item allocation of the scale according to their comprehension of the connotation of insulin medication literacy, and suggestions and the rationales were encouraged to be given. Subsequently, expurgations and revisions of items or contents have been made correspondingly.
The authority coefficient of experts were computed by their familiarity degree to the concept of insulin medication literacy and judgmental reference. An authority coefficient (Cr) of over 0.8 is considered to be acceptable[33]. For experts coordination coefficient, Kendall’s coefficients of concordance (Kendall’s W) ranges from 0.40 to 0.59 indicating a grudgingly acceptable degree of chance agreement[34]. Content validity was assessed by content validity index (CVI). Content validity index for the overall scale (S-CVI/Ave) and for each item of the scale (I-CVI) were calculated. The CVI was assessed by asking the experts to rate each item according to item’s relevance on a four-point scale, ranging from “high relevance” to “irrelevance.” Irrelevance scored one reflecting there was no correlation between the item and its belonging domain and the overall scale, and slight relevance scored two reflecting weak correlation, and relevance scored three reflecting certain correlation, and high relevance scored four indicating strong correlation. A CVI score of 0.79 or above for each item was considered to be acceptable. Items with a CVI score between 0.70 and 0.79 were revised and items with a CVI score less than 0.70 were excluded[35].
Phase 3: Pilot survey
To reduce the number of items, the 37-item Ch-InMLS was pre-tested in a total of 220 patients with diabetes mellitus. In this stage, a total of 200 completed questionnaires were collected back and examined validity. The response rate was 90.91%. Statistical analysis methods used for item selection were as follows.
Item discrimination analysis: total scores of collected questionnaires were ranked from high to low, among which 27% with low total scores from the lowest one were defined as low score group, and 27% with high total scores from the highest one were defined as high score group. each item in two groups was tested difference using independent t-test method and the items with no significant difference in scores between the two groups were excluded.
Correlation coefficient method: Pearson’s correlation coefficients between each item and the overall scale, and between each item and its belonging domain were calculated. Items with Pearson’s correlation coefficient r < 0.4 were considered as low correlation and were recommended to be removed.
Homogeneity test: If an item was deleted and a significant increase was present in the alpha coefficient, then deletion was considered. Besides, Communalities less than 0.2 was also considered to be removed.
Phase4: Formal investigation
To examine the psychometric properties of the tool, construct validity, criterion validity, as well as reliability were assessed.
Construct validity: Construct validity of this scale was assessed by exploratory factor analysis (EFA) which is usually used to generate the factor structure and confirmatory factor analysis (CFA) which is usually used to test the fit of the hypothetical factor structure. The EFA was conducted to extract factors by performing the principal components analysis with maximum variation method. The Kaiser–Meyer–Olkin (KMO) coefficient and Bartlett’s sphericity test were used to assess the suitability of the data. Then, the factor structure obtained from EFA was tested by CFA. Goodness-of-fit was evaluated by using the chi-square minimum/degree of freedom (χ2 /df), Root Mean Square Error of Approximation (RMSEA), Goodness of Fit Index (GFI), Comparative Fit Index (CFI), Incremental Fit Index (IFI), Parsimonious Normed Fit Index (PNFI) and Parsimonious Goodness-of-Fit Index (PGFI). The reasonable threshold levels of these indices for CFA were considered as χ 2 /df lower than 3, RMSEA,the value less than 0.08, GFI, AGFI, and IFI above 0.90, RMR less than 0.05[36].
The convergent and discriminant validity of the scale were assessed, and standardized factor loadings, average variance extracted (AVE), composite reliability (CR) were calculated for the final model. Convergent validity evaluates the degree of correlation of multiple items of the same domain both theoretically and practically. AVE above 0.5 and CR above 0.7 confirmed that convergent validity was satisfying. Discriminant validity indicates the level of difference between different latent variables and is valid if the average variance was greater than squared correlation coefficients[34].
Reliability analysis: The Cronbach’s alpha value was calculated to evaluate the internal consistency and reliability of the total scale and each dimension, and a value of greater than 0.7 was considered good internal reliability[37]. The test–retest reliability was used to evaluate the stability of the scale and it was measured by Pearson’s correlation coefficient in 40 randomly collected patients with diabetes mellitus 2 weeks after formal investigation. A value of correlation coefficient over time more than 0.75 (P < 0.05) was considered good test-retest reliability[38].
Scoring Criteria for Ch-InMLS: This research scale assessed insulin medication literacy level of patients with diabetes mellitus containing four domains namely knowledge, attitude, practice and skill. For item K6 to K10,P5, S2, answering right for each item scores 2, and answering wrong or answered “I don’t know” scores 1. For item K1 to K5, A1 to A11, response option of 5-point Likert scale (totally agree, agree, not sure, disagree, totally disagree) for each item, scores of 5, 4, 3, 2, 1 were applied accordingly. For item P1 to P4, P6, P7, S1, S3 to S8, response option of 5-point Likert scale (always, often, sometimes, seldom, never) for each item, scores of 5, 4, 3, 2, 1 were also used accordingly. Besides, there were eight items in the attitude domain scoring reversely. The summed total score on this 37-item scale ranged from 37 to 164, with higher scores meaning higher insulin medication literacy level.