Study design
This study is a comparative effectiveness research study aimed at detecting the influence of two different instructional approaches on the acquired practical skills during undergraduate surgical training.
Study participants
Study participants were undergraduate medical students at Goethe-University Hospital in Frankfurt in the fourth year of a 6-year program completing their obligatory surgical training. All participants were naïve regarding CMF specific practical skills and knowledge.
Participation was voluntary and took place after written informed consent, which was revocable at any time. Students were blinded towards the used instructional approaches as well as affiliation to any study group. Basic data regarding student age, sex, and duration of the study were collected using a questionnaire.
The study was conducted according to the ethical principles of the Helsinki Declaration (Ethical Principles for Medical Research Involving Human Subjects), and the local ethics committee noted that no further approval was necessary.
Study protocol
The study was carried out within the ‘training week of practical clinical skills in surgery’ [22], which aims to teach students basic surgical competencies and prepare them for their upcoming surgical rotations. This takes place at the skills lab and consists of 12 teaching units for basic general and surgical skills from all surgical disciplines. The teaching content is based on the learning objectives for practical skills defined in the national competency-based catalog of learning objectives in surgery from the German Society of Surgery [23]. The training has a capacity of 64 students per week, with a maximum of eight students per group and tutor.
On day one, students rotate through four 90-min training units of basic clinical and surgical principles. These training units are taught by peer tutors. From the second to the fifth day, students rotate through eight 210-min training units, of which five are taught by surgical specialists (general surgery, trauma surgery, vascular surgery, thoracic and heart surgery, craniomaxillofacial) and three are taught by peer tutors (punction and injection, taking medical histories, wound management).
Following the first week of practical skills training, students pass through 2 weeks of surgical rotation at the university hospital or associated teaching hospitals with participation on the ward, the ambulance, and in the operating theatre to integrate and apply the acquired surgical knowledge and skills under supervision in daily clinical practice.
Intervention
For the intervention, two fundamental clinical competences from the craniomaxillofacial surgery (CMF) spectrum, namely the performance of a structured facial examination (SFE, video at https://youtu.be/S-b3kIzmLQw) and the packing of a modified Bellocq tamponade (BT, video at https://youtu.be/gtMq4044RlM) were examined (Figure 1). These competencies were chosen since previous studies found medical students to have significant shortcomings in CMF-specific competencies [24–26]. Furthermore, the specialty was shown to be underrepresented in national undergraduate education [27], although consultations in the field are of considerably socioeconomic and numerical importance in the acute and normal care [28,29].
The existing instructional manual for the CMF unit was reworked and adapted for the study purpose for both skills and both instructional approaches. Each manual consists of a detailed schedule and workflow, as well as a step-by-step checklist to ensure a standardized sequence of the training. For further quality assurance and standardization, an instructional video was designed for both skills [30]. All instructors involved received an online-training, where both skills were demonstrated and trained in each of the instructional approaches as well as the correct performance of each instructional approach.
The instructional approaches ‘Video 4-step Approach’ as well as the existing approach ‘See One, Do One’, which was regarded as the control group, were compared.
‘See One, Do One Approach’
This approach is labeled as the main component of clinical-bedside teaching. Students learn by watching an expert explaining and demonstrating a skill. This is followed by the first independent performance of the skill, which is mostly with a patient [4]. As quality assurance and standardization for the demonstration of each skill from the trainers, each skill was videotaped based on the existing manual and checklist. The trainer demonstrated the video and explained the skill in detail, step-by-step. The explanations were predetermined in the manual and trained in the tutor training. Subsequently, students could practice the skill under the supervision and, if needed, receive correction from the tutor. Each student should practice each skill at least once. The training lasted for 60-min.
‘Video 4-step Approach’
For this study, the ‘4-step Approach’ – as described by Walker and Peyton [10] – was modified. For Step 1, the video was demonstrated without any comments. For Step 2, the trainer demonstrated the video and explained the skill step-by-step in detail. The explanations were predetermined in the manual and trained in the online-tutorial. For Step 3, the video was paused after each step, and students were chosen one-by-one by the trainer to explain the next instructional step of the video, which was then video-played. Step 4 was performed, as described by R. Peyton. Subsequently, students could practice the skills under the supervision and, if needed, receive correction from the tutor. Each student was advised to practice each skill at least once. The training lasted for 60-min.
Assignment of the students to instructional approaches of the skills
The assignment of students to one of the eight learning groups per training week with a maximum of six students per group who pass through the teaching units together occurred prior to the training week, independent of the authors and independent of study participation by the deanery. The allocation of the learning group in the study to the two instructional approaches was performed alternately.
Outcome measures
To assess the acquired competence in both skills of the study, the OSCE-format (one station for each skill) was used during the training week directly after the intervention (T1) and 5–13 (SD = 3.16) weeks later (T2), as part of the curricular and summative surgical OSCE (10 stations in total). A trinary scoring scale was used (0 points for not done, 1 point for done but incorrect, and 2 points for done and correct) for each checklist, which was based on the checklist used in the tutor manual. A time-frame of 5-min to complete each OSCE station was given. Subsequently, students received short feedback regarding their performance and suggestions for improvement at T1. During their training week, students were video-recorded (Camera System: Panasonic HC-X929) for later performance measurement and to validate the used Checklists by two independent, blinded examiners. During their surgical OSCE, due to examination regulations, the performance was not video-recorded and measured by only one examiner for each skill using the same now validated checklists. All examiners were blinded towards the students’ instructional approach and affiliation of the learning group. They received training before the OSCE to gain experience in the use of the checklist.
Data analysis
Microsoft Excel 2016 (Microsoft Office 2007, © Microsoft Corporation, Redmond, USA) for Mac and SPSS Statistics version 19 (IBM, Armonk, USA) were used for the statistical analysis and graphical display of data.
To test for significant differences in learning success between the ‘Video 4-step Approach’ and ‘See One, Do One Approach’, the Mann-Whitney-U-Test for non-parametric data was used. To test for performance differences within the respective groups at different times, the Friedman test for repeated measures was used. Furthermore, effect sizes were calculated using Cohen’s d, which is defined as the difference between two means divided by a standard deviation for the data, resulting in a unitless value that helps to interpret the effect size of observed results and hence the statistical power of a study. For most types of effect sizes, a larger absolute value indicates a stronger effect. Furthermore, it can be used as an additional control test since prior studies have shown that significant test results alone are not sufficient to interpret data and draw conclusions [31].
Additionally, each item of the checklists that were used to measure student performance was categorized into four (extraoral examination, neurological examination, midface examination, intraoral examination) or three (material preparation, insertion of the tamponade, fixation of the tamponade) subgroups. Averages from those subgroups were checked for significant differences using the Mann-Whitney-U-Test for non-parametric data.
Sample size estimation
Based on prior examination results from the years before the intervention, we estimated an average student performance of 70% with a standard deviation of 10% in the OSCE. A sample size of 88 was calculated based on the following parameters: mean ‘4-step Approach’ = 33, mean ‘See One, Do One’ = 30, SD = 10, alpha = 80%, beta = 20%.