The main focus of the developed tool called PATS was that it should be an indigenous tool, addressing the domains of Professionalism identified in the framework of ABIM in the local context. With the purpose that students can improve their professionalism as they graduate from medical college.
Key findings
This four-phase study identified domains of professionalism similar to those of the American Board of Internal Medicine. Characteristics identified are fairly similar to those as in other instrument for assessing professionalism. Indicating that these are universal characteristic of medical professionalism which are useful in teaching and assessing for learning in the local context.
The construct of respect was observed to be most important by the participants in the local context. They were of the view that it should reflect in the overall approach of a clerkship student towards patients. This element is of utmost priority in south-Asian culture and especially Pakistan with different ethnic population but a common religion which is Islam which can guide and influence the behavior of doctors. Duty should be seen in students’ activities such as completing assignments on time. Effective communication skills are very important and are needed when dealing with peers patients and seniors. The element of Honor and Integrity should be displayed by the medical students as future doctors in their interactions and especially under high -pressure situations during which it may be difficult to exhibit. In regard to excellence it was agreed that the students need to maintain a proper dress code. They should not misbehave as they are representing a respectable profession. Whereas, the construct of accountability was perceived as the least important.
Relevance to similar tools in the literature
The domains of professionalism have been identified in the literature. However, the study done by Nath did not specify domains.14 Whereas for the present study the items were developed in a more methodical manner. The study by Nath, one of the initial studies carried out on professionalism, is a survey. That study compared the perceptions of professionalism among different professional groups. The survey had 29 circumstantial statements describing behaviors of professionals on the components such as respectful, accountability, honesty and integrity, empathy self-directed learning skillful communication etc. In the present study, a focus group was conducted by the author to develop the items on the ABIM domain of professionalism, then they were validated through modified Delphi rounds. The developed tool was cognitively pre-tested and then it was piloted on a large sample of 1511 respondents. The study had three scales to rate the behaviors, but the present study has five scales to rate.
Comparing the present study with other such studies the objective was clear from the start which was to develop a tool that would be able to assess professionalism in the local context. A tool developed by Al-Eraky in the Arabian context though in a Muslim context was different in the perspective that the tool is for the assessment of professionalism of medical students and interns.15The developed tool in the present study is not intended to be used for the assessment of professionalism in interns or postgraduate trainees.
The present study is in accordance with a study carried out for the development of a tool to assess professionalism in the Taiwanese context. That study was also carried out on the 7th year, senior medical students in Taiwan by Tsai and a 34-item tool was developed.16The current study developed a 27 item tool. For the tool development, the reference panel is not specified in that study, however, the current study involved a panel of 9 participants for the focus group discussion. The methodology for piloting is not clearly given in that study, and the number of students on which it was piloted on was 133. In contrast, in the present study, the tool was first of all evaluated through pre-cognitive testing and then modifications were made and then it was pilot tested on 228 students.
The Penn state professionalism Questionnaire has 36 items and was developed with four parallel forms (Medical Students; Residents; Basic faculty and clinical faculty).17 While the present tool is targeting the clerkship undergraduate medical students for feedback purposes.
Content validity of the tool
The content validity of the tool was calculated during and after the three modified Delphi technique. Content validity index (ICV-I), Content validity scale (S-CVI) and also Content validity universal agreement was determined which was 0.94. This aspect of the study is in accordance with a tool development study to evaluate the communication skill aspect of professionalism by Zamanzadeh Vahid et al in 2015.18 The S-CVI in that study was 0.93.18 The calculated values of the final tool of the present study were of I-CVI/Avg as was 0.94 and I-CVI/UA was 0.63 which is within the acceptable range.
For the clinicians and researchers establishing the content validity of a tool is important.19 Because if it is not suitable then examining the behavior and attitude might not be reflective of the target population and the socio-cultural background.
Construct validity of the tool
In the present study there were six domains around which two to eight scales were developed. Most of the studies that have been conducted for the development of a professionalism tool confirmatory factor analysis has not been performed. 20
Adequacy of a theoretical model is judged in terms of how well it fits the observed data statistics which are: x2/d, CFI, TLI, RMSEA & SRMR. 21These indices are used to determine the assessment of fit indices and the researcher’s theoretical justification for fitness to practice. To justify the results, the fit index needs to be discussed. The basic rule of thumb is that the fit index CFI and TLI should be 0.90. 22 The author reports indices of 0.76 and 0.73 in the present study. Statistically, it is not appropriate to reject the tool. The relative chi-squared value is 0.5 which is at an acceptable range, the reason that the RMSEA values are good. Although the x2/d is less than 5, suggesting a good model fit. The low CFI and TLI are explained by the less sample size which was 228. Upon comparing the CFA results of the present study with a study by Maat et al, a good model fit can be seen. But that study was on the learning environment of high school students and had a sample size of 1887 which might be the reason for high CFI and TLI in that study.23
Reliability
The alpha reliabilities of most factors were good, except for altruism which was low but acceptable.24 One of the main reasons might have been in the understanding of the concept of altruism. The causes could have been in the translation of the concept of altruism identified during the focus group discussion.
Practical Implications
The methodological micro-scenario statement can help regulate the professional aspect of the behavior and attitude of the students towards better patient care through feedback at the undergraduate level improving the Professionalism of the students. It can be modified and used to assess the professionalism of the students in basic sciences in the earlier years of medical training.
Study limitations
Limitations that needs mentioning include a small sample size for the piloting of the tool and due to time constraints, longitudinal follow-up was not possible. As the data was collected in one institution generalizability of the tool is not well established.
Recommendation
The tool should be further piloted with increased sample size for context validation of the developed tool. To ensure generalizability it needs to be tested in other medical institutes of Pakistan. Outcome studies can be conducted to examine the effect of the developed tool on the professionalism of the students after its intervention.