Study Design
Using a pragmatic approach, we employed a mixed method design that incorporated both quantitative (questionnaires) and qualitative (focus groups) techniques.25–27 This type of research study design capitalizes on the strengths of both quantitative and qualitative data, while minimizing the shortcoming of each methodology. Furthermore, it allows the researchers to better understand the experimental results, while incorporating the participants; perspectives. The National Institutes of Health advises a mixed method approach to research “to improve the quality and scientific power of data” and to better address the complexity of issues facing the health sciences today, including the health professions education.28,29 The initial, quantitative arm of this study included a randomized controlled trial to test the effect of the teaching method on students skill acquisition and confidence and motivation in performing the fundoscopic exam. The qualitative data – medical students perceptions were collected after the preliminary didactics experiment because we assumed that quantitative research alone could not sufficiently capture the thinking processes of the study participants, that seems to influence the effectiveness of the flipped classroom approach for improving learning.30 We then compared the revised Bloom’s taxonomy levels of knowledge attained by the two groups.
Subjects
A randomized experimental design was used to compare the effects of the two practical guidance methods (flipped classroom vs traditional method) on student performance on the fundoscopic examination. The study population consisted of 104 Chiba University medical students participating in a general medicine clerkship rotation from 2018 to 2019. All 104 medical students signed an informed consent prior to enrolling in our study. Participation was voluntary and did not impact the students’ academic standing in any way.
None of the participants had a prior clinical clerkship rotation in ophthalmology. Participants were randomly assigned to either the flipped classroom groups (intervention group: n = 51) or the traditional teaching groups (control group: n = 53)
Procedure
The outline of the study design is presented in Fig. 1. Before starting the funduscopic training, all participants examined the eye fundus on a simulator (EYE Examination Simulator®; Kyoto Kagaku Co, Kyoto, Japan) using a PanOptic ophthalmoscope (pretest). Each participant was assigned 3 cases and observed 1 eye for 90 seconds. They presented their findings thereafter and presented their findings (3 findings each).
The students in the flipped classroom groups watched a 10-minute e-learning video on fundus examination skills. They were also able to watch the video repeatedly on their smartphone, tablet, and PC. Students in the traditional teaching group, attended a lecture with a similar content as the video. In both the video and the lecture, the faculty instructed the students in each group about their grip, posture, procedure, angle, and light intensity and taught fundus examination skills using the iExaminer system while sharing screens with students. The iExaminer system consists of three core components: the PanOptic ophthalmoscope, the iExaminer adapter, and the iExaminer application.31 The PanOptic ophthalmoscope addresses the fundamental challenge in ophthalmoscopy – to get a good view of the fundus in order to make a sufficient assessment. Patented Axial PointSource™ Optics make it easy to enter undilated pupils, offering a 25º field of view, resulting in a view of the fundus ‘’that’s 5X greater than you see with a standard ophthalmoscope in an undilated eye. Direct viewing of the fundus through the PanOptic provides better images of the retinal changes caused by hypertension, diabetic retinopathy, glaucoma, and papilledema to enable clinicians to make these diagnoses earlier. The iExaminer adapter is designed to attach the PanOptic ophthalmoscope to the iPhone and the iExaminer app gives the users the ability to take, store, retrieve and send fundus images right on their iPhone. It also allows medical students and their teacher to share the same visual perspective. All instruction time was standardized to 30 minutes. Three different teachers were randomly assigned to each session. The teachers were previously trained using the same instructional guide (developed by KS, SS, and YH) in order to minimize the individual variability in guiding the students.
After completing their specific training sessions, all the study participants examined the eye fundus again (posttest) on the EYE Examination Simulator® using a PanOptic ophthalmoscope. Each participant was assigned 3 cases and observed 1 eye for 90 seconds. They presented their findings thereafter.
This study followed the Consolidated Standards of Reporting Trials (CONSORT) reporting guideline, and the flow diagram is available (Supplement 1).
Main outcome measures
The diagnostic accuracy of funduscopic findings (i.e., normal fundus, optic disc edema, pathological optic disc cupping, or not observed) and the time taken to identify funduscopic findings were assessed in pretest and posttest.
Secondary outcome measures
An anonymous, self-administered paper questionnaire was employed to assess study participants’ confidence in performing funduscopic examinations and their motivation for performing funduscopic examinations before and after their respective training sessions. Each question was answered on a five-point Likert scale (from 1 - Strongly disagree to 5 - Strongly agree). The questionnaire was developed by KS, SS, and KN based on prior published research and focus group discussions.32
Statistical analysis
We performed descriptive and bivariate analyses to describe our sample. The diagnostic accuracy in funduscopic findings, time taken to identify funduscopic findings, as well as the confidence and motivation for performing funduscopic examinations between the two groups were compared by using the two-way analysis of variance (ANOVA). All statistical analyses were performed using IBM SPSS version 26.0 (IBM Corp. Armonk, NY).
Focus group
A qualitative inquiry was conducted following the quantitative study to help explain the quantitative results. A sample of 36 medical students was randomly selected from the quantitative study participants (18 from the flipped classroom group and 18 from the traditional classroom group).33 After obtaining informed consent, we conducted six focus groups lasting about 30 minutes, each considering workflow impacts and participants’ fatigue. Groups were organized separately with the students who participated in the flipped classroom approach (3 groups, n = 18) and those who participated in the traditional classroom (3 groups, n = 18).
Trained moderators (KS and DY) asked open-ended questions about students’ perception of the effectiveness of flipped classroom and traditional teaching of fundus examination (diagnostic accuracy and the time taken to identify funduscopic findings). They asked about what went well and what did not go well in the educational session, and the significance of the flipped classroom approach (Supplement 2). The focus groups were recorded and transcribed verbatim. The transcripts were analyzed using deductive content analysis drawing upon the revised Bloom’s taxonomy as the coding frame, with the cognitive process dimensions as the theme categories and subcategories.34,35 Open coding of the focus groups transcripts was done by KS and DY. The authors independently read and coded all transcripts, and then discussed, identified, and agreed on the coding of the descriptors. Inter-rater degree of agreement between two researchers was assessed using the Cohen’s kappa statistics.
KY derived theme categories and subcategories as they emerged from the data. To ensure credibility of the findings, the theme categories and subcategories were regularly discussed with and reviewed for content by DY, who has extensive experience in qualitative research.18 Concepts for each of the cognitive process dimension in the revised Bloom’s taxonomy 16 were analyzed, and the number of units of analysis for each concept was counted (Supplement 3). We have grouped similar codes into a theme and then checked to see which dimension of the cognitive process it corresponded.