Evaluating the Impact of a Web-Based Gamified Approach on Pharmacology Education in Medical Students

doi:10.21203/rs.3.rs-3934551/v1

Download PDF

Research Article

Evaluating the Impact of a Web-Based Gamified Approach on Pharmacology Education in Medical Students

https://doi.org/10.21203/rs.3.rs-3934551/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Background

The safe use of drugs is global health concern, and good prescribing depends on the adequate undergraduate training of medical students. However, the extensive theoretical content of each drug, limited motivating didactic strategies and time constrains hamper proper training to minimize medication errors.

Methods

This study aimed to assess the usability of a web-based game designed for the non-traditional teaching of pharmacology, intending to stimulate guided competence among medical students. Additionally, the research aimed to compare the improvement in knowledge acquisition after the game's use. The usability of the web game was initially evaluated by 11 medical students from the University of Antioquia using the Mobile App Rating Scale. Subsequently, the academic performance of 21 students from the same university was assessed before and after using the digital tool.

Results

The usability test, involving 4 women and 7 men with an average age of 20 years, revealed an overall usability score of 4.09. Impressively, 92% of respondents recommended the application, and 50% expressed a willingness to pay for access. Evaluation of academic performance demonstrated a significant improvement in the percentage of correct answers obtained in an academic test using the web game compared to scores obtained prior to its use (median 90% vs 70%, P value = 0.023).

Conclusion

The use of a web game designed to stimulate guided competence holds promise for enhancing the learning experience in pharmacology.

Gamification

software development

Innovation

Pharmacology

Medicine

Medication errors account for about 42 billion dollars in global costs annually, equivalent to 1% of global health expenditure. In addition to the monetary cost, these errors may cause psychological and physical pain and suffering from the request to the drug's consumption [1]. These incidents contribute to a decrease patient satisfaction and a decrease in the trust on the health system [1, 2].

Prescribing errors by health professionals include incorrect drug selection, therapeutic duplication and mistakes in doses, routes of administration or dosage forms. These errors are preventable through solid education in pharmacology during undergraduate academic training [3]. The World Health Organization, in its Guide to Good Prescribing, highlights that undergraduate clinical training tends to focus more on diagnostic capacities than on therapeutic ones, resulting in inappropriate prescriptions by newly graduated doctors in almost half of the cases [4].

In daily practice, pharmacology professors face the challenge of having students internalize the fundamental concepts of rational drug use [4]. This challenge arises in part due to the extensive theoretical content of each drug and the limited dedication of the students, despite the strategies implemented, such as the use of an electronic vademecum, work in small groups and clinical rounds with polymedicated patients [5]. It is often observed that many students forget much of the information taught in previous semesters [6].

Experience can be an effective form of lasting learning, but in high-risk situations such as prescribing medications, errors could compromise patient safety [6]. Since the 1960s, video games have been used in military and medical training, providing a realistic simulation without risk for participants [7]. Since then, many educational digital games have been developed, most of them aimed at basic education. Since “digital natives” learn better through digital games and tools that promote competition [8, 9], there is a need for an innovative approach for university students.

In this context, the potential of the game as a teaching and learning tool was explored through the evaluation, using the Mobile App Rating Scale (MARS), of a web game designed for the non-traditional teaching of pharmacology [10]. The objective was to assess usability through aspects such as functionality, aesthetics, privacy, and integration, to stimulate guided competence among medical students and to compare the improvement in knowledge acquisition.

Programming language and personnel involved.

For the development of the web game, the following tools were used: (i) Database: SQL managed by Postgres, (ii) Backend: Elixir managed by Phoenix, (iii) Frontend: HTML5, Css and Javascript managed by Vue.js, (iv) Design: Adobe suite, specifically Illustrator and XD. The application was designed 100% responsive. The development of the software was carried out in 8 months. The work team consisted of seven people, four of them involved in database programming, backend, frontend and graphic design, respectively, the others in the elaboration of user requirements, objectives, supervision and drafting of questions and answers. The project was managed through Gitlab. The game was called FOGUEO (the acronym for Operational Pharmacology: Online Student Warfare in Spanish).

Test users and setting of the study.

Fifth semester undergraduate medical students enrolled in the Faculty of Medicine of the University of Antioquia.

Questions and answers.

They were designed by physicians with graduate degrees on pharmacology or toxicology who belong to the Faculty of Medicine of the University of Antioquia. The questions were based on the contents that are traditionally taught in the pharmacology course during the fifth semester of the medical curriculum, and the relevance for clinical practice, as derived from the Therapeutic Optimization Program for Multimedicated Individuals with High Health Risk (OPTIMMAS+), which is implemented at the Alma Mater Hospital in Medellín, Colombia.

The questions were designed to be answered through multiple selection with a single answer. Each question had six answers, one true and five false. The questions were designed to be shown to the players randomly; of the six answers, only four were offered at a time, one true and three false, the order of the answers was also aleatory.

Content.

The game consists of four worlds related to the major areas that must be covered in the pharmacology course of the fifth semester, these are: (1) Overview of pharmacokinetics and pharmacodynamics, (2) Drugs that act on Receptors, (3) Drugs acting on ion channels, transporters, enzymes, and nucleic acids, and (4) Anti-infective drugs.

In turn, the questions from each world were distributed according to four approaches: (1) generalities, (2) high-risk drugs, (3) interactions and (4) clinical cases. Players could choose one of these approaches or use a random question scheme in the game.

Development of the game.

Each game consists of five questions filtered according to the player's selection and the world's choice. Each question has 40 seconds to be answered, an incorrect answer or the completion of the stopwatch indicates an error in the answer. A game is won with 3 correct answers.

Winning a game awards 1 point to the player, a lost game does not award points. Three points allow the player to move to the second world, six points to world three, nine points to world four and twelve points allow her to win the game.

Usability validation.

Using Google Forms, players rated the application according to the MARS scale [10] which evaluated commitment, functionality, aesthetics, information, security, service provision and privacy. The scale contained 13 ordinal questions and 13 nominal questions. The ordinal questions were rated from 1 to 5, with 1 being the most negative result and 5 being the most positive. The nominal questions contained the criteria yes, no and maybe and were rated 5, 0, 3 respectively. The answer doesn't apply or I don't know were rated as 1. An objective global average of more than 3 was considered positive.

Sample size.

To calculate the sample size needed to ensure that the interface met the usability requirements, the analysis proposed by Cazañas et al. was used [11]. Therefore, 11 users were required to detect 80% of the problems with the interfaces, with a maximum overestimation ratio of 30% and a mean root square error of only 0.05.

For the comparison, the improvement in knowledge acquisition, a paired non-probability sample of 18 users was calculated. Assuming that the mean of the differences in scoring is 1 and the standard deviation of the differences was 1.35, one would have 80% power and a level of significance of 5% to affirm that the mean of the paired differences is significantly different from zero [12]. However, since it was not possible to ensure a normal distribution, and non-parametric tests were required, the sample size needed for a parametric test was taken and added 15%, as some authors recommend [13], resulting in 21 paired subjects.

Evaluation of academic performance.

The students answered anonymously through Google Forms a pre-test of general pharmacology at the middle of the academic semester. Those selected as the intervention group had access to the game for at least 2 months. In parallel, 5 students who attended the standard course but had no access to the platform were included as a control group. At the end of the academic semester, a post-test was carried out on all users.

Data analysis.

The results of the pre and post evaluations were compared using the non-parametric Wilcoxon sign ranges test. A P-value < 0.05 was considered as a significant difference.

In Fig. 1, a block diagram detailing the structure of the application is presented. The players accessed through the IP address: http://34.69.166.25:82/#/, where they registered using their institutional emails. These emails were previously entered into the database to validate access permissions. The login process required both a username and a password.

Each player started in the first world with zero points, progressing through the worlds by winning games to complete the game.

Figure 2 shows various views of the graphical interface. During login, the player's level and score were displayed (panel A). Locked worlds were identified with a padlock, while those already completed were marked at the bottom of the world (panels B and C). The selection of the parameter was indicated by the color of the pulsed frame, specific to each world (image D). Correct answers were highlighted in green, while incorrect answers were shown in red (panel E). The wallpapers and the avatar varied depending on the player's progress.

The usability of the game was evaluated by 11 medical students (4 females and 7 males, with an average age of 20 years). All participants had at least basic knowledge of managing software, such as social networks and email. Table 1 presents the results broken down by item. The provision of services (i.e., e-commerce) and privacy was the only criteria that attained a score lower than 3. The overall usability score was 4.09, and 92% of respondents recommended the application, while 50% would be willing to pay to access it.

Table 1

Score obtained during FOGUEO validation using the MARS scale.
Quality Criteria	Item	Mean score per item	95% Confidence Interval (lower to upper)	Mean score by criteria
Commitment	Entertainment	4.64	4.30 to 4.98	4.26
	Interest	4.46	3.99 to 4.92
	Personalization	3.73	2.68 to 4.77
	Interactivity	3.82	3.09 to 4.54
	Audience Adequacy	4.64	4.30 to 4.98
Functionality	Performance	3.91	3.21 to 4.61	4.18
	Ease of use	4.82	4.55 to 5.00
	Navigability	3.73	2.82 to 4.63
	Gestural Design	4.27	3.67 to 4.88
Aesthetics	Bottoms	4.64	4.30 to 4.98	4.43
	Graphics	4.46	4.10 to 4.81
	Visualization	4.18	3.78 to 4.59
Information	Exactitude	5.00	5.00 to 5.00	4.58
	Objectives	4.55	3.53 to 5.00
	Quality	4.09	2.73 to 5.00
	Visual	5.00	5.00 to 5.00
	Evidence	3.40	1.92 to 4.88
	Ethics	5.00	5.00 to 5.00
	Credibility	5.00	5.00 to 5.00
Security	Transparency	4.55	3.53 to 5.00	3.31
	Updates	3.91	2.99 to 4.83
	Risk Management	1.46	0.24 to 2.87
Provision of services and Privacy	E-commerce	1.64	0.16 to 3.12	2.30
	Data protection	2.45	0.80 to 4.11
	Bandwidth	2.82	1.42 to 4.22
Global score	Overall usability score	4.09	3.73 to 4.45	4.09

Figure 3 presents the results of evaluating the academic performance of 21 medical students. Compared to the score obtained prior to the use of the technology (pre-test), there was a statistically significant improvement (P = 0.0225) in the percentage of correct answers obtained in the academic test using the web application.

In the digital age, teaching methods and means are necessarily being transformed. This is due to the loss of interest of contemporary students in traditional learning environments due to greater exposure to social networks, multiplicity of platforms and the widespread use of intelligent devices [14]. Here we present the implementation and evaluation of a web game for learning pharmacology in medical students, which generated promising and revealing results both in terms of the usability of the game and its effectiveness in improving academic performance.

The concept of gamification began to be used in 2008 in the business world [15] and denotes the use of techniques, elements and dynamics typical of games in non-recreational activities, in order to enhance motivation and reinforce behaviors aimed at solving a problem and improving productivity [16]. Despite how recent the concept is, its popularity has grown rapidly, in part due to the expectation of solving the great challenge of making learning interesting and engaging in traditional teaching environments.

Modern pedagogical strategies include collaborative, problem-based learning, computational thinking, among others. Although not traditionally considered primary educational material, games are gaining recognition for their effectiveness in medical education.

Gamification can activate motivation for learning, allow for feedback, facilitate meaningful learning by being more attractive, and build student loyalty to the process. In addition, it allows measurable results when competing for levels, points and medals, generates autonomy and develops competitiveness without losing collaboration thanks to connectivity between users in the online space [17], this may explain the results observed in our study. Another great benefit of gamification is the ability to simulate a situation that recreates a sense of risk. This mechanism has been useful in financial education. Students are trained in realistic environments that simulate a stock exchange, which generates a sense of risk necessary to know what it is like to work under pressure, but without losing real money, and also promotes the user engagement facilitating basic and psychological need satisfaction [18]. For university students, the use of technology is natural and comfortable, and learning through this medium is more attractive to them than through classrooms [19].

The MARS scale provides a multidimensional measure of the objective and subjective quality indicators of the applications [10]. The validation of FOGUEO made it possible to establish that provision of services and privacy are two elements to improve, although e-commerce is not the aim of our game. The evaluators considered that the questions and answers of the game are of high quality, in addition, that the game is fun and entertaining, however, with minor deficiencies in customization and interaction. Willig et al. consider that reward mechanisms such as scores, badges, leaderboards, medals, and level changes are the elements that generate the most motivation in players [20]. In our application we use scores and level changes, future versions may include those other aspects mentioned by Willig et al [20].

According to a systematic review of the use of digital games in the education of students in the health area, at least five studies demonstrated that such games had a positive effect, of moderate to large magnitude, when comparing the score obtained in knowledge tests after their use versus those students exposed only to traditional learning. In ten studies, there was evidence of a small to large impact on skill improvement compared to traditional learning. The difference seems to be smaller when comparing digital education with the use of games, but still with a trend in favor of games [21].

Another systematic review published in 2012 identified a total of 25 articles describing a total of 30 games used to train technical and non-technical skills relevant to the surgical field, claiming for validation before its integration into surgical teaching curricula [22]. In 2013, Willig et al. launched a multiple-choice online question game, called Kaizen, to evaluate the clinical knowledge of their Internal Medicine students. They evaluated information retention, acceptance, and withdrawal rate. Participants identified the leaderboard as the main motivation to participate [20]. Something similar was done at the University of Valparaiso, Chile, where a mobile application with internal medicine topics was evaluated. Significant improvement in performance was found among students assigned to the mobile application (average increase of 14.5 ± 8.9 vs 9.4 ± 11.6 points, P value = 0.03). A reduction was also found in the total time and the average time per question, which was significant in the analyses of complete cases (P value = 0.04) [23].

Notwithstanding, applications for teaching pharmacology are rare, although from a theoretical point of view, knowledge tends to be more extensive for learning in this field than in surgical areas. Although they did not use digital strategies, Vallejos et al. of the University Health Sciences Foundation in Bogotá, Colombia, have described the gamification of teaching pharmacology using didactic strategies such as crossword, focus, stop, and other analogues of classic games, demonstrating student satisfaction of more than 70% with improvement in retention and motivation after use [24]. In addition, Ormazabal Valladares et al. reported the use of role-playing for teaching pharmacology in nursing students, through a randomized controlled experimental study, students exposed to role-playing achieved significantly higher scores compared to control groups and general satisfaction with the development of these methodologies [25]. Finally, Avenzoar, a multiplatform game from the University of Seville, Spain, was the only accessible application found in the literature similar to FOGUEO, after a search carried out in August 2023 using Google Scholar, PUBMED and gray literature in the Google search engine, using Spanish or English as a languages and the combination of keywords such as “Game, Pharmacology, Application, ICT, Learning”. This application allows competition of pharmacists against friends or against random users, generates a ranking according to the score obtained and offers prizes for attending events in pharmacies. Its interface is user-friendly and although it offers more than 10,000 questions on the subject and Google Play reports in August 2023 that it has more than 5,000 downloads and 80 ratings, which give it a 4-star score, with negative comments generally related to functionality and positive comments extolling commitment; according to Girardello et al, rating applications using stars provides little or no significant information about their quality [26], it is therefore necessary to validate using scales such as MARS.

It is important to note the limitations of the study, such as the voluntary, non-random participation in the evaluation of academic performance could introduce biases. Software development applied to pharmacology education is new, so the budget allocated to this type of project in universities is usually low, considering that personnel for the scientific and technical construction of these must be highly specialized, so a significant investment in human talent is required to support it, which can be a constraint. Additionally, the questions, both in the game and in the pre and posttest were not validated. No psychometric analysis was done. These limitations offer opportunities for future research and improvements in the design and application of the game.

Over the past years, the world has learned to move virtually. The pandemic forced education to seek teaching alternatives through technology. According to our results, gamification in pharmacology can improve students' academic performance. Web applications are tools that can be included in medical courses and allow students to demonstrate their knowledge in an alternative way to lectures, seminars or workshops. In the coming years, gamification can be designed for teaching pharmacology in different areas of healthcare, even in postgraduate studies.

MARS, Mobile App Rating Scale

FOGUEO, Operational Pharmacology: Online Student Warfare (acronysm in Spanish)

OPTIMMAS+, Therapeutic Optimization Program for Multimedicated Individuals with High Health Risk

Ethics approval and consent to participate. The need for ethical approval was waived but informed consent was obtained from every included participant.

Consent for publication. Not applicable.

Availability of data and materials. The web game, datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Competing interests. The authors declare no competing interests related with this study.

Funding. This work was supported by University of Antioquia (grant 2022-49030).

Authors' contributions. AFZ played a key role in the conception, design, supervision, analysis, interpretation, and the creation of the software. Additionally, AFZ substantively revised the work. LMRV, HJP, MAMG, and CAR were involved in designing the application's contents and acquiring and interpreting data. LMRV and HJP were responsible for drafting the work. All authors have given their approval for the submitted version of the manuscript to be published. Furthermore, all authors have agreed to be personally accountable for their individual contributions and to ensure that any questions related to the accuracy or integrity of any part of the work, even those in which the author was not personally involved, are appropriately investigated, resolved, and documented in the literature.

Acknowledgements. We thank A.F. Botero Ospina for developing the frontend and Deysy Bedoya for designing the game. To the medical students at the University of Antioquia who kindly agreed to test the game and participate in the study. To Gabriel J Restrepo, professor at the University of Antioquia, for his contribution during the development of questions and answers.

Tariq RA, Vashisht R, Sinha A, Scherbak Y: Medication Dispensing Errors and Prevention. In: StatPearls. edn. Treasure Island (FL); 2024.
Maurette P, Sfa CAMR: To err is human: building a safer health system. Ann Fr Anesth 2002, 21(6):453-454.
Bos JM, van den Bemt P, de Smet P, Kramers C: The effect of prescriber education on medication-related patient harm in the hospital: a systematic review. Br J Clin Pharmacol 2017, 83(5):953-961.
Tichelaar J, Richir MC, Garner S, Hogerzeil H, de Vries TPGM: WHO guide to good prescribing is 25 years old: quo vadis? Eur J Clin Pharmacol 2020, 76(4):507-513.
Likic R, Maxwell SR: Prevention of medication errors: teaching and training. Br J Clin Pharmacol 2009, 67(6):656-661.
Tobaiqy M, McLay J, Ross S: Foundation year 1 doctors and clinical pharmacology and therapeutics teaching. A retrospective view in light of experience. Br J Clin Pharmacol 2007, 64(3):363-372.
Bergeron B: Developing Serious Games (Game Development Series): Charles River Media, Inc.; 2005.
James C, Davis K, Flores A, Francis JM, Pettingill L, Rundle M, Gardner H: Young people, ethics, and the new digital media. Contemporary Readings in Law and Social Justice 2010, 2:215+.
Prensky M: From Digital Natives to Digital Wisdom: Hopeful Essays for 21st Century Learning. In. Thousand Oaks, California; 2012.
Stoyanov SR, Hides L, Kavanagh DJ, Zelenko O, Tjondronegoro D, Mani M: Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth 2015, 3(1):e27.
Cazañas A, de San Miguel A, Parra E: Estimating Sample Size for Usability Testing. Enfoque UTE 2017, 8(1):pp. 172-185.
Statulator: An online statistical calculator. Sample Size Calculator for Comparing Two Paired Means. [http://statulator.com/SampleSize/ss2PM.html]
Lehmann EL, D' Abrera HJM: Nonparametrics : statistical methods based on ranks, Rev. 1. ed edn. New York: Springer; 2006.
Limniou M: The Effect of Digital Device Usage on Student Academic Performance: A Case Study. Education Sciences 2021, 11(3):121.
Walz SP, Deterding S: The Gameful World: Approaches, Issues, Applications: The MIT Press; 2014.
Zichermann G, Cunningham C: Gamification by Design: Implementing Game Mechanics in Web and Mobile Apps: O'Reilly Media, Inc.; 2011.
Borras-Gene O: Fundamentos de gamificación; 2015.
Bitrián P, Buil I, Catalán S: Enhancing user engagement: The role of gamification in mobile apps. Journal of Business Research 2021, 132:170-185.
Shatto B, Erwin K: Teaching Millennials and Generation Z: Bridging the Generational Divide. Creat Nurs 2017, 23(1):24-28.
Nevin CR, Westfall AO, Rodriguez JM, Dempsey DM, Cherrington A, Roy B, Patel M, Willig JH: Gamification as a tool for enhancing graduate medical education. Postgrad Med J 2014, 90(1070):685-693.
Gentry SV, Gauthier A, L'Estrade Ehrstrom B, Wortley D, Lilienthal A, Tudor Car L, Dauwels-Okutsu S, Nikolaou CK, Zary N, Campbell J, Car J: Serious Gaming and Gamification Education in Health Professions: Systematic Review. J Med Internet Res 2019, 21(3):e12994.
Graafland M, Schraagen JM, Schijven MP: Systematic review of serious games for medical education and surgical skills training. Br J Surg 2012, 99(10):1322-1330.
Martínez F, Tobar C, Taramasco C: Implementation of a Smartphone application in medical education: a randomised trial (iSTART). BMC Medical Education 2017, 17(1):168.
Vallejos Narváez ÁL, Sergio Iván; Ávila, María Paula; Ruano, Camilo Andrés; Maldonado Arenas, Sebastián; Merchán, Johan Sebastián; Valdés, Sergio Alejandro; Latorre Montoya, Germán.: Didácticas innovadoras en el aprendizaje de la farmacología en estudiantes de medicina / Innovative teaching in pharmacology learning for medicine students. Rev Hosp Ital B Aires 2004, 36(4):99-106.
Ormazábal Valladares V, Almuna Salgado F, Hernández Montes L, Zúñiga Arbalti F: Juego de roles como método de enseñanza de Farmacología para estudiantes de la carrera de enfermería. Educación Médica 2019, 20(4):206-212.
Girardello A, Michahelles F: AppAware: which mobile applications are hot? In: Mobile HCI: 2010; 2010.

No competing interests reported.

Download PDF

Version 1

posted

You are reading this latest preprint version

Evaluating the Impact of a Web-Based Gamified Approach on Pharmacology Education in Medical Students

Status:

Version 1

Abstract

Background

Methods

Results

Conclusion

Figures

BACKGROUND

METHODS

Data analysis.

RESULTS

DISCUSSION

CONCLUSIONS

LIST OF ABBREVIATIONS

DECLARATIONS

REFERENCES

Additional Declarations

Status:

Version 1