In our study, TUS, a national, external ranking examination-comprising MCQs only, is strongly correlated, with medium to large effect sizes, with students’ grades at medical school. Although TUS does not specifically assess the demonstration of clinical skills, the results of TUS are strongly correlated with the school results that incorporate tests of practical competence as well as cognitive skills. This may be due to the underlying factors (intelligence, conscientiousness etc) that determine performance. Although this study demonstrates that a student’s performance across all domains of assessment in the undergraduate program is a good predictor of cognitive skills in an external national examination in the early post graduate phase we can not comment on the correlation with early postgraduate practical competence. Had the TUS included tests of practical clinical competence it might not have been surprising to find that after a year-long internship, students’ skills had accelerated. Whether that rate of development could be predicted by their total undergraduate performance remains unknown in our context and should be explored in future projects.
Previous similar studies, mainly from North America, use United States Medical Licensing Examination (USMLE) as the criterion variable [4–6]. There are also examples from the Netherlands [7] and Australia [8]. All report similar findings with those from North America. This study helps to generalize the findings about the correlation between undergraduate performances with an external exam internationally. However we also found that students’ performance across a medical program is (fairly) consistent in keeping with the findings of Hope and Cameron [9] and McManus et al [10]. McManus et al. identified that the continuity of academic success for medical students stretched from secondary school into the early years of their postgraduate careers and referred to this as ‘the academic backbone’
Although GPAs from all years have moderate to large correlations with TUS, there is a trend for the correlation power to increase from 1st to 4th years. Year 1 has the lowest correlation of all. This may be partly because some subjects in the 1st year curriculum such as history, foreign language, Turkish language etc. are not directly related to medical sciences and are not reflected in the TUS. The low correlation may also reflect the effect the adaptation period has on some students in an unpredictable fashion. The 4th year medical school GPA is the best predictor of TUS scores (r = 0.67). This may reflect the alignment between the medical school curriculum and the focus of the TUS; in this first clinical phase of education year 4 students have the main clinical clerkships such as Internal Medicine, General Surgery, Gynecology/Obstetrics and Pediatrics. The high correlation with year 4 may also reflect the development in students’ professional identity and thus their motivation to study. Fourth year may also be the year when all students’ study strategies become more focused to plan their careers for specialization. In this study the 4th year GPAs correctly identified students at risk of failing TUS. If this finding is ratified with further cohorts it will provide an evidence-informed metric that the medical school can use to detect and support those students who may need further remediation before they proceed further. The 5th year GPAs are the highest and the grade range narrows to 1,55. This might be due to the checkpoint function of 5th year. As students require a cumulative GPA over 2.00 to pass to the final year; they may be preparing hard for the exams in order to lift their grades.
What message should findings in this study give to policy makers in Turkey and internationally? In the current system, TUS is the only assessment that determines which students enter specialization (residency) training. If TUS is strongly correlated with medical school performance, does this indicate that TUS is a valid and reliable method to select candidates for each residency? There were some students who outperformed their peers in TUS although they had a lower school performance. Such a final, end-of-school exam may be identifying some students who have accelerated their performance during their internship due to improved motivation, a preference for learning in the workplace, unknown factors or a combination of these. On the other hand TUS may be criticized for its focus on MCQs only. In contrast, medical school performance could be considered a better measure of global achieved learning using a range of assessment tools that evaluate problem solving, communication and practical skills [11]. In addition, the medical school’s use of multiple methods allows for compensation of each tool’s weaknesses [1]. At this stage while the TUS continues as an MCQ only examination it is difficult to determine if the unexplained variance between school and TUS performance is due to the difference in learning outcomes addressed by different assessment tools or by differences in achieved learning over the internship.
It is reassuring that there is a good correlation between year GPAs suggesting that they may each be useful additional determinants for selection to residency slots.
The results suggest that for students in the medical school under study, adding the undergraduate performance to the TUS results may increase the validity of the decisions about the allocation of residency posts. However it cannot be assumed to be so across medical schools. In order to be more confident about graduates’ competence and the selection processes, studies looking at further cohorts and across a range of medical schools is required.
A major strength of this study is being able to include all students with available data in the analysis. As the student data were available longitudinally, consistency of student success could also be evaluated. With a strong inter-correlations of year specific GPAs we can claim that student success has a high degree of consistency. This study also has some limitations. It is based in one-institution and analyzes student outcomes from one year of graduation. Further work should cross-validate the results by involving different institutions and graduates from different years. Another factor is that, students make extensive preparations for TUS, which may limit our comments for direct effects of school curriculum on TUS. We couldn’t analyze the gender effect on student performance as assigning gender codes to un-identified students could jeopardize the confidentiality of data due to small numbers in the study. Although we have demonstrated that medical school performance is a good predictor of performance in the early graduate phase our study was not designed to explore the causal mechanisms. Do hard-working successful students become hard-working successful graduates? Are the exam results in medical school or in the TUS a result of commercial preparation courses? Does learning in the internship year differ from that in the first 5 years? Why do students fail the TUS? These interesting questions, raised by this analysis, will require a more qualitative approach to address them.