A Comparative Study Between Traditional Face-To-Face, Audience Response System, and A Flipped Classroom Plus Audience Responce System in Pharmacology Education.

doi:10.21203/rs.3.rs-131282/v1

Download PDF

Research Article

A Comparative Study Between Traditional Face-To-Face, Audience Response System, and A Flipped Classroom Plus Audience Responce System in Pharmacology Education.

https://doi.org/10.21203/rs.3.rs-131282/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Background: Active learning methods, audience response system (ARS) and flipped classroom (FC), have applied in higher education internationally. However, these outcomes in Japan are uncertain. This study presents a comparison of teaching methods such as the traditional face-to-face (TFF), ARS, and FC plus ARS that are used to enhance student pharmacists’ understanding of pharmacology.

Methods: Sophomores at Ohu University were recruited for this study from 2017 to 2019. The TFF were performed in 2017; the ARS were performed in 2018; and the FC plus ARS were performed in 2019. Exam scores were compared between the three methods and anonymous questionnaires were also surveyed in 2018 and 2019 to compare students’ preferences. The Kruskal-Wallis test and ANOVA were carried out depending on data distributions. Categorical data were analyzed using the Chi square test and the Fisher’s exact test. The ANCOVA was performed if background information between the three cohorts was significantly different.

Results: The groupings of the participants went as follows: 49 students in 2017, 78 students in 2018, and 90 students in 2019. Their academic backgrounds of each year at the end of their freshman year were similar except for their grade point averages (GPAs). The medians of their pharmacology exam correct rates for the TFF, ARS, and FC plus ARS were 73.3%, 90.0%, and 93.3%, respectively. ANCOVA utilizing GPA as the covariance revealed that the ARS and the FC plus ARS significantly improved exam correct rates compared with the TFF (p < 0.001). There was no significant difference between the ARS and the FC plus ARS groups (p = 0.487). A subgroup analysis in terms of gender difference revealed that the ARS was a significantly improved method compared to the FC plus ARS for female students (p = 0.009), whereas the ARS was not for male students (p = 0.819). Anonymous questionnaires revealed that students in the FC plus ARS lectures had good readiness, enjoyed their classes, and felt fulfillment compared with the ARS lectures.

Conclusions: Active learning methods including the ARS and FC plus ARS are significantly more effective than TFF methods.

Educational Philosophy and Theory

Internal Medicine

flipped classroom

audience response system

active learning

pharmacology

In 2012, active learning was academically defined by the Ministry of Education, Culture, Sports, Science and Technology (MEXT) with this statement:

It is important for students with little motivation and purpose to take the attitude of independently studying. Universities have to reexamine the teaching method, such as by providing opportunities to participate in interactive classes and active activities. “Expected approaches to the university” include the development of interactive learning, the active use of teaching assistants and student assistants, and a small number of students. It illustrates the promotion of guidance and utilization of information and communication technology. (MEXT)

Therefore, some universities in Japan have tried to adjust their classes based on this statement.

A flipped classroom (FC) and an audience response system (ARS) have been included in active learning through the utilization of information and communication technology. Some papers have shown positive learning performances for student pharmacists using these teaching methodologies (1-6). Conversely, a few negative articles also have been reported (7,8). Other academic field articles regarding the FC have been internationally reported (9-13). Recently, a meta-analysis for FC in comparison with traditional face-to-face lectures was published and the FC improved students’ learning (14). Comprehensively, the FC seems to be a positive way of teaching for student pharmacists. In Japan, few reports on active learning written in Japanese have been published; however, most of them have reported students’ preference for active learning. Only an article from Nishigawa et al. compared the FC with team-based learning (TBL) at a school of dentistry in Japan mentioned that there was no statistical difference in the examination score between the FC and TBL, and both styles are more effective than traditional style lectures and constitute valid formats for clinical dental education (15) Thus, active learning outcomes in Japan are uncertain. Therefore, the purpose of this study was to examine which teaching method is the most effective teaching method for student pharmacists, such as traditional face-to-face (TFF) lectures, ARS lectures, and FC plus ARS (FC plus ARS) lectures.

Targeted students

This study targeted sophomore students taking part in the pharmacology course offered at Ohu University School of Pharmaceutical Sciences from 2017 to 2019. The pharmacology course consisted of 13 categories from their sophomore year to the 4^th grade. Topics lectured were taught as 4^th category in the sophomore year along with endocrine systems; moreover, this included the hypothalamus, pituitary abnormality, adrenal cortical hormone, adrenal medulla hormone, and adrenal cortical dysfunction. These topics were lectured twice a week for two weeks (1.5 hrs /class, 6 h) in the fall quarter. The TFF lectures were performed in 2017; the ARS lectures were carried out in 2018; and the FC plus ARS lectures were performed in 2019. Scores of external exams and prerequisite grade point averages (GPAs) in the freshman year were collected to obtain academic background information regarding sophomore students in 2017, 2018, and 2019. Prerequisite subjects included chemistry, biology, physics, mathematics, introductory chemical thermodynamics, basic science exercises, basic pharmaceutical training, biochemistry, functional morphology, organic chemistry, basic analytical chemistry, and physical chemistry.

Learning outcomes

The pharmacology course placed an emphasis on being knowledge-based, so the expected outcomes were that sophomore students fully understood the concept of endocrine systems.

Audience response system (ARS)

The ARS lectures were carried out using Clica^TM (http://clica.jp/LP/). This active learning tool collected the students' understanding, reactions, and opinions in real time during a lecture; subsequently, all of the posts can be displayed on the students’ smartphones as well as on a screen in the class. Students can view items and collaborate with each other; this activity can also happen between the lecturer and the students. This is so they can understand the lecture from various perspectives while comparing it with their own opinions. The lecture explained each topic in the class, and multiple-choice questions were administered to the students through Clica^TM during the lecture. They responded as quickly as possible and the lecturer and the students viewed their responses on the screen and the lecturer added extra explanations depending on the students’ incorrect responses using handouts of the topics.

Flipped classroom (FC) plus ARS

Six-hour lectures/four classes were flipped. Prerequisite videos were uploaded on YouTube beforehand, and the URLs of each video were sent to students’ emails at least three days before the lecture. Video times for the first, second, third, and fourth video were 10 min, 16 s; 10 min, 50 s; 13 min, 16 s; and 10 min, 18 s, respectively. Data regarding the video views and watch times were retrieved from YouTube Analytics. During the class, multiple-choice questions were administered to students through Clica^TM at the beginning of each lecture. The lecturer and the students viewed their responses on the screen, and the lecturer added extra explanations depending on the students’ incorrect responses using handouts of the topics.

Primary outcome

A primary outcome of the study was to compare the correct rates of exams for endocrine systems in the pharmacology course between three ways of teaching such as the TFF lectures, the ARS lectures, and the FC plus ARS lectures. An exam consisted of 30 multiple-choice questions. Identical questions were administered to each of the three classes’ cohort but multiple choices were completely reshuffled every year. A subgroup analysis was performed in terms of gender difference.

Anonymous questionnaire survey

Anonymous questionnaire surveys were conducted at the last lecture on the endocrine systems in the pharmacology course in 2018 and 2019. This was done in order to examine the students’ preference for the ARS and the FC plus ARS.

Ethical considerations

This study was approved by the ethics committee of Ohu University (No. 220). The lecturer issued a letter to the students to inform them of the purpose of this study at the end of the last lecture. Written informed consent was obtained from those students who were willing to participate in this study. All methods were performed in accordance with the relevant guidelines and regulations.

Statistics

The sample size for the study was calculated using G*Power software; subsequently, the total sample size was calculated as 159. This figure was arrived at by assuming the use a one-way analysis of variance (ANOVA) for statistical analysis with an effect size of 0.25, alpha error probability of 0.05, power of 80%, and the number of groups being set a three.

The Kruskal-Wallis test was carried out for nonparametric distribution data; duly, a one-way ANOVA was carried out for parametric distribution data. Categorical data were analyzed using the Chi square test and the Fisher’s exact test. The ANCOVA was performed if background information between the three cohorts was significantly different. P values less than 0.05 were considered statistically significant. All statistical analyses were performed with EZR (“Easy R”) (Saitama Medical Center, Jichi Medical University, Saitama, Japan), which is a graphical user interface for the programming language, R (The R Foundation for Statistical Computing). More precisely, it is a modified version of R commander designed to add statistical functions that are frequently used in biostatistics (16).

Background information regarding the sophomore students in 2017, 2018, and 2019 are shown in Table 1. Notably, the total number of included students exceeded 159 and gender differences in each year were not significantly different. External exam scores in the fall of 2019 were not significantly different. Additionally, the prerequisite GPA for freshman year in each year was not significantly different, except for 2018.

Exam scores for pharmacology are compared in Figure 1. The medians of correct rates in 2017, 2018, and 2019 were 73.3%, 90.0%, and 93.3%, respectively (p < 0.001 for 2017 vs. 2018; p = 0.029 for 2018 vs. 2019; p < 0.001 for 2017 vs. 2019). Since the prerequisite GPA in 2018 was significantly different in comparison with that in 2017 and 2019, the ANCOVA with GPA as covariance was carried out, as shown in Figure 2. There was a significant difference between the three ways of teaching methods (p < 0.001). Furthermore, there was no significant difference between the ARS and the FC plus ARS groups (p = 0.487).

Since a prerequisite GPA in terms of gender difference was not a covariance, a subgroup analysis for the correct rate/GPA between the three ways of teaching in terms of gender difference was performed, as shown in Figure 3. Regarding the female students, the ARS was the most effective method in the three ways of teaching (median: TFF 26, ARS 39, FC plus ARS 33, p < 0.001). Conversely, regarding the male students, the ARS and the FC plus ARS were significantly different than the TFF, but not significantly different between the ARS and the FC plus ARS (median: TFF 25, ARS 35, FC plus ARS 31, p < 0.001). A Bartlett’s test of homogeneity of variances revealed that the variances by the ARS between female and male students were significantly different (p = 0.020), whereas those by the FC plus ARS were not (p = 0.532).

The students’ response rates for both the ARS lectures and the FC plus ARS significantly decreased over time [(1^st: 84.7%, 3^rd: 76.5%, and 4^th: 68.1%; p < 0.001 for one-way ANOVA for the ARS lectures) (1^st: 87.8%, 2^nd: 80.0%, 3^rd: 75.0%, and 4^th: 66.5%; p < 0.001 for one-way ANOVA for the FC plus ARS)]. The 2^nd response rates for the ARS lectures were not obtained because of technical issues.

The numbers of video views and watch times regarding the pre-lecture and post-lecture for the FC plus the ARS are shown in Table 2. Video views for pre-lectures gradually decreased; however, the length of the watch time depended on the topic. Some student pharmacists viewed videos again for review after the lectures.

The results of the anonymous questionnaire survey regarding the ARS and the FC plus ARS are shown in Figure 4. Students’ readiness, enjoyment, fulfillment, and feeling to talk to other people regarding the FC plus ARS lectures were significantly better evaluations than the ARS (p < 0.001, p = 0.019, p = 0.006, and p = 0.005, respectively).

This study examined which teaching method is the most effective one for student pharmacists. Accordingly, the methods that were considered were the TFF, ARS, and the FC plus ARS. As a result, the ARS and FC plus ARS, which are called “active learning methods,” were effective teaching methods in comparison with the TFF. Subsequently, our research strongly indicates there are no comparative studies regarding teaching methods between the ARS and the FC plus ARS. This study presented the conclusion that there was no significant difference between the ARS and the FC plus ARS. Since GPA was not a confounding factor for correct rates in terms of gender difference, correct rate/GPA was evaluated (Figure 3), suggesting that at least the ARS is an effective teaching method for endocrine systems in the pharmacology course at Ohu University.

Some articles positively reported that the ARS is an effective educational method in classrooms (17-25). Regarding the retention of learned topics, Rubio et al. reported that long-term retention was maintained until three months later (18). However, Doucet et al. mentioned that one-year post exam scores were significantly decreased (19). A systematic review reported by Hussain et al. concluded that the ARS improved student recall immediately following an educational activity, but the effects were not lasting (21). Accordingly, the ARS educational effect is for short-term understanding.

In addition, there have been positive articles regarding FC (3, 14, 26). Although this study demonstrated that the FC plus ARS was significantly more positive than the TFF, it was not more significant than the ARS. Additionally, a gender difference was found between the ARS and the FC plus ARS and so there are several reasons to consider this result. As Table 2 shows, the number of video views gradually decreased. Unfortunately, since YouTube analytics was not able to distinguish viewers from others, the faculty members were not able to identify who in fact watched videos or not. Some articles pointed out gender differences in active learning methods (27); nevertheless, they do not speculate our results. To be sure, variances of correct rate/GPA by the ARS for male students were significantly different in comparison with the female students because of the significantly lower GPAs of the male students in relation to the female students. This could be because there was no significant difference between the ARS and the FC plus ARS for the male students.

Response rates to the ARS in the classes were significantly decreased as time went on both the ARS lectures and the FC plus ARS lectures. Probably, most students are familiar with using the ARS in the classes and they would have feel bored. Therefore, some ingenuity to maintain high response rates with the ARS would be necessary for students’ academic performance. Therefore, a Jeopardy-style review game would be useful to overcome this problem (28).

Both the ARS and the FC plus ARS were new teaching methods for the student pharmacists at Ohu University. Both methods effectively worked to increase the students’ motivation to learn (Figure 4). Readiness for the FC plus ARS group was significantly higher than that for the ARS group. This is reasonable because student pharmacists were required to watch the topical videos before attending lectures. In addition to this, some student pharmacists watched videos from the post-lectures to the date of their exams (Table 2), suggesting that the FC videos would effectively review topics concerning independent study at home by watching the videos again. Moreover, the FC plus ARS ones were enjoyable and provided a sense of fulfillment for the student pharmacists. The most important point to support the results would be that the short videos were approximately 10 to 15 minutes in length. Some papers with positive results mentioned that the students’ preference was that the length of the videos were long and the subsequent increase to their workloads from watching them led to unsatisfied feelings.(1, 5)

One interesting article by Fryer reported that incentives offered for educational outputs—such as better grades—are less effective than those for educational inputs which can include attendance, good behavior, or wearing uniforms (29). Of the positive result articles regarding the FC, Giuliano’s article was successful in light of this concept (4). In a course grade composition, the pre-quiz weight was increased from 7.5% to 15% to emphasize the importance of preparation prior to class. The current study did not offer any incentives to student pharmacists for educational inputs and outputs. Therefore, some incentives presented to student pharmacists for educational inputs would work much better in understanding topics in pharmacology course.

This study had some limitations, with the first one being the costs for preparing the FC videos were not evaluated. Although creating video files with PowerPoint software and using YouTube and Clica^TM were free, the workloads for preparing the videos such as recording explanations and inserting them into a PowerPoint file took too much time. Furthermore, this study was performed by only two lecturers so that expanding this teaching method to all faculty members would not be guaranteed because of a lack of an evaluation of the time consumption. A way of thinking about using teaching assistants would be a good option to solve this limitation (30). On the other hand, it was reported that faculty time to implement a FC required 127% more time than in the previous year’s traditional lecture format. Yet, that time was expected to decrease once the course was established (26). Then, although the current study elucidated short-term effects to keep fundamental knowledge-based contents by the active learning methods, the long-term effects were uncertain. Therefore, examinations of their long-term effects would be necessary for future academic strategies.

This study examined which teaching method is the most effective one for student pharmacists. The methods considered in this regard were the TFF, the ARS, and the FC plus ARS. In conclusion, the ARS and the FC plus ARS, which are called active learning methods, were the most effective teaching methods for endocrine systems in pharmacology course.

The Ministry of Education, Culture, Sports, Science and Technology (MEXT)

Traditional face-to-face (TFF)

Audience response system (ARS)

Flipped classroom (FC) plus ARS (FC plus ARS)

Team-based learning (TBL)

Grade point averages (GPAs)

Analysis of covariance (ANCOVA)

One-way analysis of variance (ANOVA)

Ethics approval and consent to participate

This study was approved by the ethics committee of Ohu University (No. 220).

Consent for publication

Not applicable.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Competing interests

The authors declare no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Authors' contributions

NN created the study design, collected data, and was a major contributor in writing the manuscript. TY made an arrangement of pharmacology course for sophomore students and finalized the manuscript.

Acknowledgments

We owe a very important debt to Prof. Hayasaka, who provided technical help and encouragement. The presence of any errors is of course my responsibility; moreover, I am also responsible for the final formulation, and any errors that may concern it are entirely mine.

Kugler AJ, Gogineni HP, Garavalia LS. Learning Outcomes and Student Preferences with Flipped vs Lecture/Case Teaching Model in a Block Curriculum. Am J Pharm Educ. 2019;83:7044.
Lichvar AB, Hedges A, Benedict NJ, Donihi AC. Combination of a Flipped Classroom Format and a Virtual Patient Case to Enhance Active Learning in a Required Therapeutics Course. Am J Pharm Educ. 2016;80:175.
Koo CL, Demps EL, Farris C, Bowman JD, Panahi L, Boyle P. Impact of Flipped Classroom Design on Student Performance and Perceptions in a Pharmacotherapy Course. Am J Pharm Educ. 2016;80:33.
Giuliano CA, Moser LR. Evaluation of a Flipped Drug Literature Evaluation Course. Am J Pharm Educ. 2016;80:66.
Wong TH, Ip EJ, Lopes I, Rajagopalan V. Pharmacy students' performance and perceptions in a flipped teaching pilot on cardiac arrhythmias. Am J Pharm Educ. 2014;78:185.
González A. Turning a traditional teaching setting into a feedback-rich environment. Int J Educ Technol High Educ. 2018;15:32.
Bossaer JB, Panus P, Stewart DW, Hagemeier NE, George J. Student Performance in a Pharmacotherapy Oncology Module Before and After Flipping the Classroom. Am J Pharm Educ. 2016;80:31.
Khanova J, McLaughlin JE, Rhoney DH, Roth MT, Harris S. Student Perceptions of a Flipped Pharmacotherapy Course. Am J Pharm Educ. 2015;79:140.
Flores Ò, del-Arco I, Silva P. The flipped classroom model at the university: analysis based on professors’ and students’ assessment in the educational field. Int J Educ Technol High Educ. 2016;13:21.
Kurihara Y. Flipped Classroom: Effects on Education for the Case of Economics. J Educ Elearn Res. 2016;3:65-71.
Nouri J. The flipped classroom: for active, effective and increased learning – especially for low achievers. Int J Educ Technol High Educ. 2016;13:33.
Pattanaphanchai J. An Investigation of Students' Learning Achievement and Perception using Flipped Classroom in an Introductory Programming Course: A Case Study of Thailand Higher Education. J Univ Teach Learn Pract. 2019;16:Article 4
Wozny N, Balser C, Ives D. Evaluating the Flipped Classroom: A Randomized Controlled Trial. J Econ Educ. 2018;49:115-29
Hew KF, Lo CK. Flipped classroom improves student learning in health professions education: a meta-analysis. BMC Med Educ. 2018;18:38.
Nishigawa K, Omoto K, Hayama R, Okura K, Tajima T, Suzuki Y, et al. Comparison between flipped classroom and team-based learning in fixed prosthodontic education. J Prosthodont Res. 2017;61:217-22.
Kanda Y. Investigation of the freely available easy-to-use software 'EZR' for medical statistics. Bone Marrow Transplant. 2013;48:452-8.
Cain J, Black EP, Rohr J. An audience response system strategy to improve student motivation, attention, and feedback. Am J Pharm Educ. 2009;73:21.
Rubio EI, Bassignani MJ, White MA, Brant WE. Effect of an audience response system on resident learning and retention of lecture material. AJR Am J Roentgenol. 2008;190:W319-22.
Doucet M, Vrins A, Harvey D. Effect of using an audience response system on learning environment, motivation and long-term retention, during case-discussions in a large group of undergraduate veterinary clinical pharmacology students. Med Teach. 2009;31:e570-9.
Mostyn A, Meade O, Lymn JS. Using Audience Response Technology to provide formative feedback on pharmacology performance for non-medical prescribing students--a preliminary evaluation. BMC Med Educ. 2012;12:113.
Hussain FN, Wilby KJ. A systematic review of audience response systems in pharmacy education. Curr Pharm Teach Learn. 2019;11:1196-204.
Gauci SA, Dantas AM, Williams DA, Kemm RE. Promoting student-centered active learning in lectures with a personal response system. Adv Physiol Educ. 2009;33:60-71.
FitzPatrick KA, Finn KE, Campisi J. Effect of personal response systems on student perception and academic performance in courses in a health sciences curriculum. Adv Physiol Educ. 2011;35:280-9.
Liu FC, Gettig JP, Fjortoft N. Impact of a student response system on short- and long-term learning in a drug literature evaluation course. Am J Pharm Educ. 2010;74:6.
Mostyn A, Meade O, Lymn JS. Using Audience Response Technology to provide formative feedback on pharmacology performance for non-medical prescribing students--a preliminary evaluation. BMC Medical Education. 2012;12:113.
McLaughlin JE, Griffin LM, Esserman DA, Davidson CA, Glatt DM, Roth MT, et al. Pharmacy student engagement, performance, and perception in a flipped satellite classroom. Am J Pharm Educ. 2013;77:196.
Liaw S-S, Huang H-M. How Factors of Personal Attitudes and Learning Environments Affect Gender Difference toward Mobile Distance Learning Acceptance. Int Rev Res Open Dis. 2015;16:104-32.
Cusick J. A Jeopardy-Style Review Game Using Team Clickers. MedEdPORTAL. 2016;12:10485.
Fryer RG. Financial Incentives and Student Achievement: Evidence from Randomized Trials. Q J Econ. 2011;126:1755-98.
Spangler J. Costs related to a flipped classroom. Acad Med. 2014;89:1429.

Table 1. Background Information regarding Sophomore Students in 2017, 2018, and 2019.

Year	2017	2018	2019	p value
Method	Traditional face-to-face (TFF)	Audience Response System (ARS)	Flipped classroom plus ARS (FC+ARS)
Students (Female/Male)	49 (31/18)	78 (45/33)	90 (52/38)	0.795*
External Comprehensive Examination: Median* (Freshman Year: Fall)	0.90	0.87	0.89	0.315**
Chemistry: Median	0.95	0.82	0.90	0.108**
Math / Physics: Median	0.78	0.91	0.86	0.335**
Biology: Median	0.99	0.89	0.87	0.118**
Prerequisite GPA: Median (Freshman Year)	2.7	2.1	2.7	＜0.001**
Female (Median)	2.8	2.2	2.9	0.015**
Male (Median)	2.4	1.8	2.8	＜0.001**

*Chi-square was used to determine significance, defined as p < 0.05.

** Kruskal-Wallis test was used to determine significance, defined as p < 0.05.

***Students’ scores were divided by the whole average score.

Table 2. Video Views and Watch Time before Lectures and after Lectures for FC plus ARS Lectures.

		Pre-lecture		Post-lecture to Exam
	Video Time	Video Views	Watch Time	Video Views	Watch Time	Audience Retention　(%)
1st	10 min, 16 s	133	9 min, 12 s	41	4 min, 06 s	44.4
2nd	10 min, 50 s	106	7 min, 06 s	28	2 min, 54 s	43.1
3rd	13 min, 16 s	84	10 min, 18 s	21	2 min, 54 s	56.4
4th	10 min, 18 s	50	5 min, 30 s	25	2 min, 54 s	66.9

1st Hypothalamus, Pituitary abnormality

2nd Adrenal Cortical Hormone, Adrenal Medulla Hormone

3rd Adrenal Cortical Dysfunction (1)

4th Adrenal Cortical Dysfunction (2)

Download PDF

Version 1

posted

You are reading this latest preprint version

A Comparative Study Between Traditional Face-To-Face, Audience Response System, and A Flipped Classroom Plus Audience Responce System in Pharmacology Education.

Status:

Version 1

Abstract

Figures

Introduction

Methods

Results

Discussion

Conclusion

List Of Abbreviations

Declarations

References

Tables

Status:

Version 1