New EPA-based psychiatry clerkship curriculum
A clerkship curriculum based on the PROFILES catalogue was developed. Ten students participated in the new EPA-based psychiatry curriculum and spent four weeks (approximately 50 hours per week) on ten different wards. The core EPAs with nested psychiatry clerkship EPAs are shown in Table 1. Students were introduced to the clerkship structure in a clerkship orientation seminar at the first day of the clerkship. In addition to working on the wards and taking part in routine ward and department meetings, students participated in six one-hour didactic clerkship seminars (on psychopathology, psychiatric interventions, old-age psychiatry, psychosis, stress-related disorders, personality disorders, a scientific paper presentation exercise and a scientific journal club). Furthermore, they participated in off-site visits to specialized addiction treatment facilities, which included a patients-as-teachers seminar and the child and adolescent psychiatric clinic.
KPM Level 1 educational outcome (satisfaction): Students’ evaluation of the clerkship experience
Overall satisfaction with the clerkship was rated with a 5-point Likert agreement scale to the statement “I am very satisfied with the overall clerkship experience” (1 = disagree, 5 = agree) and resulted in an overall average of 4.9 (range: 4.0-5.0). Seminars and off-site visits were rated with grades (1 = bad, 5 = excellent) and received an average grade of 3.5 (range: 2.0-5.0). Students felt well integrated in their ward teams (4.8, range: 4.0-5.0 on a 5-point Likert scale) and found the online learning platform as helpful for their learning experience (4.5, range: 4.0-5.0 on a 5-point Likert scale). They weakly agreed on whether workplace-based assessments were helpful for their learning experience (3.6, range: 2.0-5.0 on a 5-point Likert scale). Students mentioned motivation of attending physicians and residents, as well as structure and organization of the clerkship curriculum as positive aspects in their written evaluations. They emphasized the importance of structured bed-side teaching in the first clerkship week and suggested better instruction of residents with regards to workplace-based assessments as well avoiding redundancy in some of the face-to-face seminars.
KPM Level 2 educational outcome (learning): Students’ self-assessment of need for supervision regarding EPAs before and after the clerkship and learning reflections
Each student rated his or her need for supervision for nine nested psychiatry clerkship EPAs in the beginning of the clerkship (asked on first day) and the end of the clerkship (asked on last day) on a 6-point Likert scale (range from 1 = “I can only observe this activity”, to 2 = “I can do this only as a co-activity with the supervisor”, 3 = “I can do this activity, if the supervisor is present”, 4 = “I can do this, if the supervisor completely repeats the activity”, 5 = “I can do this, if the supervisor repeats the important parts of the activity”, 6 = “I can do this, if I can ask for help when I need it”). Changes differed across EPAs and intraindividually. The average change of supervision need for all nine EPAs was 2.9, which is equivalent to moving from observing an EPA to doing an EPA independently and having a resident check the activity. However, the change in supervision level ranged from 1 to 6. Single EPA change (except of EPA 9) and overall change of self-assessed need for supervision was statistically significant (Wilcoxon signed rank test, p < 0.05). Results are shown in Figure 1.
With regards to personal learning reflections, each student documented four patient admissions including admission diagnosis and personal learning effect. The full range of psychiatric disorders was covered in the clerkship. Documented admissions included schizophrenic disorders (n = 11, 28%), affective disorders (n = 10, 25%), substance abuse disorders (n = 5, 13%), anxiety and stress related disorders (n = 5, 13%), personality disorders (n = 3, 8%), organic illness (n = 1, 3%), eating and impulse control disorders (n = 1, 3%), and two admissions did not contain a diagnosis (5%). The vast majority of written reflections addressed various aspects of communicating with psychiatric patients (e.g. “adjust communication strategy to a patient who is logorrheic and long-winded, in such a case use short and precise questions early on” or “learned how to deal with a patient who doesn’t want to answer a question and refused a physical exam, communicate acceptance”). Fewer comments addressed specific aspects of the mental status exam and planning and structuring a patient admission.
KPM Level 3 educational outcome (behavioral change): Workplace-based assessments of EPAs in the clerkship
Clinical residents supervised the students and completed 63 workplace-based assessments (average of 6.3 per student). In week one 30% (n=19) EPA assessments were documented, in week two 41% (n=26), in week three 17% (n=11), and in week four 8% (n=5), two assessments were not dated (3%). The medical faculty provided an official 3-point-Likert scale for supervision (1= activity possible with help from supervisor, 2 = activity possible with minimal help from supervisor, 3 = activity possible with indirect supervision). The average EPA supervision levels as evaluated by clinical supervisors per week were: 2.5 (week 1), 2.6 (week 2), 2.5 (week 3), and 3 (week 4). Of all assessments, 38% (n=24) addressed EPA 1, 43% (n=27) addressed EPA 2, 3% (n=2) addressed EPA 3, 1.5% (n=1) addressed EPA 5, and 14% (n=9) addressed EPA 8. EPAs 4, 6, 7 and 9 were not used for workplace-based assessments. Students only partly perceived these assessments as relevant for individual learning progress (average = 3.6, SD = 0.8 on a 5-point Likert scale).
With regards to written feedback, 97% of assessment forms contained written feedback. All feedback forms were signed off by residents, except one which was signed off by a nurse. No assessment form was signed off by attending physicians, psychologists or social workers. Written feedback was predominantly formulated as a short positive affirmation of what was done well (e.g. “structured mental status examination” or “good communication during physical examination, empathic attitude”). Only few written feedbacks also contained actionable areas of future improvement (e.g. “try to interrupt logorrheic patient next time” or “add disorder specific questions when interviewing relatives, such as memory function in suspected dementia”). Comparing and contrasting written feedback per student indicated that supervisors in some cases referred back to previous feedbacks and validated progress (e.g. from “practice written summary of mental status” to “precise summary of mental status, pay attention to relevant information for other team members, such as housing, financial situation and family situation).