Study design
The present study relies on a sequential exploratory mixed methods design for development of an instrument. In a sequential exploratory design, the first step is to collect and analyze qualitative data; next, based on the results of the analysis of the qualitative data, the quantitative phase of the study where quantitative data are collected and analyzed begins; finally, the results of the qualitative and quantitative analyses are interpreted together [16]. According to Creswell (2012), one of the primary functions of a sequential research design is the development of instruments. The development of a new instrument for evaluation of a clinical concept necessitates a careful consideration of its complexity and various dimensions; thus, initially, the components of the concept in question should be identified and the structural dimensions of it should be clearly established [17]. In the qualitative stage, the conventional content analysis approach was used to determine triage nurses’ perceptions of the concept of professional capability, identify relevant concepts, and develop items. Employed in many nursing studies, qualitative content analysis is a research method for mental analysis of the content of text data through the processes of systematic classification, assigning codes, and making themes or designing familiar models [18]. A definite advantage of conventional qualitative content analysis is the direct acquisition of clear data from the study free from the bias of previously-known subjects or theories [19]. In the present study, in addition to using a qualitative approach, the researcher conducted an extensive literature review to further verify the items. The validity of the questionnaire was measured in terms of face validity, content validity, and construct validity.
Sample and setting
In the first stage of the present study (the qualitative phase), data were collected using personal interviews, focus interviews, and observation. Accordingly, 24 in-depth, semi-structured interviews were conducted in which 20 nurses (18 triage nurses and 2 triage head nurses), 2 general practitioners, and 2 emergency medicine specialists were interviewed face-to-face. In addition, two focus interview sessions were held with a group of 5 triage nurses and the researcher carried out 48 hours of observation. The inclusion criteria for the nursing staff were having a bachelor’s degree in nursing and at least one year of professional practice; the emergency doctors were required to be at least a G. P. and have 6 months’ experience of work in triage. Both nurses and doctors were included only if they were willing to participate. For ethical considerations, The Institutional Review Board of the researchers’ university has verified that the study complies with research ethics (decree code: IR.SUMS.REC.1396.S197). Before the interviews, the participants were informed about the objectives of the study, the voluntary nature of their participation, methods of data collection and why the interviews were to be recorded, the roles of the researcher and the participants, confidentiality of their information, and anonymity of the participants. Subsequently, they were asked to sign an informed consent form if they were willing to participate in the study. The participants were also informed that they were free to withdraw at any point of the research and the time of the interviews would be set by their agreement. In order for the observations to be ethical, the participants were observed with prior notice and in an overt manner.
Assessment of face and content validity
The qualitative face validity of the questionnaire was measured thus: 15 triage nurses and emergency specialists were interviewed face-to-face and the difficulty level (difficulty in understanding the statements and terms), relevance (relationship between the items and the different dimensions of the questionnaire) and ambiguity (possibility of misunderstanding the items or unclear terms) were assessed. After the faulty items had been revised, the quantitative method of item impact testing was employed to determine the quantitative face validity of the questionnaire and the significance of each item so that the unsuitable items could be identified and eliminated. Accordingly, 15 experts were asked to score each item on a 5-point Likert scale: 5 = Very important; 4 = Important; 3 = Fairly important; 2 = Not very important; 1 = Not important at all. Subsequently, the item impact score of each item was calculated.
Content validity was measured both quantitatively and qualitatively. In the qualitative stage, 15 experts who were familiar with development of instruments and nursing were asked to examine the questionnaire in terms of syntax, use of proper vocabulary, necessity, significance, placement of the items, and scoring. The quantitative assessment of the content validity addressed content validity ratio (CVR), content validity index (CVI), and scale-level content validity index S-CVI/Ave. To measure CVR, the experts ranked each item on a 3-point Likert scale: Necessary, Useful but not necessary, Unnecessary. According to Lawshe’s table, items whose numerical value of CVR is above 0.49 are kept [20].
Evaluation of CVI was conducted according to Waltz and Bausell’s index: the relevancy, clarity, and simplicity of each item were ranked on a 4-point Likert scale by 15 experts [21]. The S-CVI/Ave of the questionnaire was calculated based on the mean of the CVI scores of all the items. According to Polit and Beck (2006), a score of 0.90 or above is acceptable for S-CVI/Ave [22]. Item analysis was performed prior to factor analysis. The object of item analysis was to determine the Cronbach’s alpha and initial reliability and identify the items that affected the reliability of the questionnaire. Item analysis is also intended to examine the relationship between the correlation coefficient of items: if an item does not have a correlation coefficient of at least 0.2–0.3 with at least another item, it is eliminated [23] ; also, if the correlation coefficient of an item with another item is above 0.7, either one of them is eliminated or they are merged. Items whose total correlation coefficient score is below 0.3 can be omitted [24]. Most studies suggest a sample size of 30 to 50 subjects for the purpose of item analysis [25]—in the present study, sample size was set at 40 subjects. An evaluation of the reliability of the questionnaire based on item analysis yielded a Cronbach’s alpha of 0.79.
Assessment of construct and divergent validity
Construct validity was determined using factor analysis. The recommended sample size for factor analysis is 5 to 10 subjects per item of an instrument. Some authorities consider 3 subjects per item as adequate provided that percent variance is reported and factor loading is above 0.80 [26]. In the present study, the number of selected participants was 10 times the number of the items of the questionnaire (350 nurses). The construct validity of the questionnaire was measured using exploratory factor analysis, Kaiser-Meyer-Olkin(KMO) index test, Bartlett’s test of sphericity, analysis of the major indexes, and Vorimax rotation. After the calculation of the correlation matrix between the variables, factors were extracted. The factor loading of every item in factor matrix and rotation matrix must be at least 0.4 [27]. In the present study, a factor loading of 0.4 was taken as the least acceptable degree of correlation between each item and the extracted factors. Evaluation of divergent construct validity was conducted using not only the developed triage nurses’ professional capability questionnaire, but Liu’s Competency Inventory for Registered Nurses (2007) [28]. Both questionnaires were distributed simultaniously among 100 triage nurses; subsequently, the correlation between the scores was analyzed.
Assessment of reliability
The reliability of the questionnaire was assessed in terms of internal homogeneity and consistency. The internal homogeneity of the instrument was measured by calculation of its Cronbach’s alpha. In the present study data collected from 350 triage nurses. A Cronbach’s alpha of 0.7 to 0.8 indicates satisfactory and adequate internal homogeneity for an instrument [29]. The consistency of the instrument was tested using the test-retest method for each factor ad the entire questionnaire. Grove et al. (2014) suggest an interval of 2 weeks to 1 month between the two tests [30]. In the present study, the interval was 20 days and the subjects’ scores from the two tests were compared using intraclass correlation coefficient (ICC) test. If the ICC index of an instrument is above 0.80, its consistency is considered as satisfactory [31]. The sample used to determine the consistency of the questionnaire consisted of 50 triage nurses who completed the questionnaire twice with a 20-day interval.