Study design
Over the span of the first four months of the research (February 3 to June 5 2019), a total of 65 students were expected to pass the orthopedic course without using the Orthobox application who comprise the control group. Each internship course lasts two weeks and includes an average of nine students; during the four months, a total of seven internship courses took place which means a total of around 65 students pass the orthopedic course. Accordingly, fourteen different standard exams were designed by the orthopedic standard exam board before study recruitment began, and seven of them were randomly chosen for our seven groups of 9 students. Then, each group was given one of these seven exams randomly at the end of the two-week course. In a similar manner, after two months of study pause (this will be more clarified in the limitation section), 65 students were expected to participate in the orthopedics course over the span of the second four months of the study (August 5 to December 18 2019) who are considered as the case groups. However, this time, the Orthobox application, available in iOs®- or Android®-based operating systems, was installed on the students’ smartphones. As such, they used the application during the course, and were then tested by the same seven final exam tests given to the control groups. The application panel provided the executors with case group activity reports for further correlational analysis. Both the case and the control group participants were asked to fill out an attitude Visual Analogue Scale (VAS) questionnaire at the end of each course before receiving their scores. Data were also collected from students’ scores on the final exams specifically designed for this research. These data were then gathered and compared with each other in order to identify the impact of the Orthobox on students’ knowledge of orthopedics.
Tools
Orthobox software: As mentioned before, the interventional tool used for this study was an application specifically designed for teaching orthopedics named Orthobox. The Orthobox application is free of charge. The app consists of five main parts of educational information (Fig. 1A): 1) frequently used medication in orthopedic wards, 2) order samples for common orthopedic hospitalized patients, 3) prescriptions for common orthopedic ambulatory patients, 4) cast and splint types, and 5) educational movies. Here is the detailed description of each part:
1) The first section is mainly a list of several groups of medication including antibiotics, anticoagulants, analgesics, fluid therapy, vaccines, and casting materials. In each group, we have introduced those medicines which are most frequently ordered or prescribed in orthopedic wards or clinics. The available information about each medicine in this application includes name and classification, forms, indications, contraindications, caution, side effects, and group in pregnancy (Fig. 1B). 2) In the second section, we have selected eight common order notes in orthopedic wards, which include orders for the following orthopedic conditions: active bleeding wound, amputated finger, fat embolism, open fracture, laceration without active bleeding, supracondylar fracture, trochanteric fracture, and deep vein thrombosis (DVT). For each order, a typical history of the patient is provided along with their X-ray or real limb image, and also the appropriate order is written at the end. Certain words in the texts of this section are highlighted, which will provide additional information if you click on them (Fig. 1C). 3) The third section presents several frequently used prescriptions for patients with orthopedic conditions such as electromyography, knee physiotherapy, radiology, and serologic tests (Fig. 1D). 4) The section about casts and splints provides specific descriptions along with their related images about different types of casts and splints, which are used in different parts of the body (Fig. 1E). 5) In an attempt to familiarize students with the casting and splinting skills, useful educational movies about upper and lower limb casting and splinting are provided in the last section of the application. This part of the application requires internet connection to work (Fig. 1F). We are going to make the app publicly available once the investigation is complete. An English-language demonstration video of the application is provided as appendix to the paper (Additional file-1). The development of a new platform enables executors to update and customize the content of the app in order to meet the demands of each specific center.
Application activity report
Each student of the case group is submitted to the application with a defined username and password which would be expired after two weeks at the end of the course. The Orthobox panel enables the executors to have access to the number of times each user visits each part of the application. At the end of the study, we had an activity report from students in case group so that we could analyze the correlation between student’s exam/ VAS score and his/her activity on application (Additional file-2).
Final exam: Both the teaching plan and final exams for orthopedics internship courses are based on 4 divisions: Order, Prescription, Medication and Skill. The final exams are designed by two designated professors from the orthopedics department. The exam material for both the control and case groups was the Textbook of Orthopedics and Fractures by Dr B. Aalami Harandi, et al., which students are supposed to self-study. For the purposes of this study, the case groups were also given access to the app as a supplementary source of study. The exam consists of four questions requiring the students to write down a complete order for a certain patient history provided on the exam sheet, write a clinic patient prescription for a presented patient, answer questions regarding a certain medicine, and finally explain the techniques of casting or splinting. Before recruitment, the orthopedics exam board prepared fourteen different, but equally difficult (the level of difficulty was carefully balanced by the in-charge professors), questions for each of the four divisions. These questions were then randomly put together to design 14 written question sheets seven of which were finally selected for students’ final exam in both groups. The exam papers were corrected based on a system of differentiating between major and minor faults. First, the writers carefully defined the standard answer, major fault and minor fault which are reported in Table 1 in detail. This table is based on similar definitions used in driving tests. The term ‘major fault’ is defined here as those answers which miss a necessary point or are completely wrong, whereas the term ‘minor fault’ refers to those answers in which the required point is mentioned but they are not the standard answer. Then, one professor (who was not part of the research team) corrected the exam papers based on these determined guidelines. Therefore, there was no bias in this regard. This method enabled us to compare the major and minor faults on different exam questions between case and control students, and to analyze their correlation with the number of visits case students had made to each corresponding part of the application.
Table 1
Major and minor fault in exam answers
Definition
|
|
Major
|
Minor
|
Definition
|
Answers which miss a necessary point or are completely wrong, leading to a complication for patients if happens in real.
|
Refers to those answers in which the required point is mentioned but they are not the standard answer.
|
Example
|
|
Major
|
Minor
|
Order
|
Q: Tibia fracture
A: Not ordering for check of the compartment syndrome signs.
|
Q: Tibia fracture
A: Not ordering the ankle and knee joints radiography
|
Prescription
|
Q: Wound healing discharge
A: adult dosing for a child patient.
|
Q: Wound healing discharge
A: Writing “Tab” instead of “Cap” for Cephalexin
|
Medication
|
Q: Heparin
A: Wrong dosing
|
Q: Heparin
A: Not writing complete medicine pharmaceutical forms
|
Procedure
|
Q: Short leg cast
A: Wrong foot angle during casting
|
Q: Short leg cast
A: Wrong but near to correct number of cast layers
|
Attitude questionnaire
A short VAS scale questionnaire (consisting six question) was designed (Additional file-3). The questions required participants to rate how much they had learned about the different branches of orthopedics knowledge, five questions about medicine, order, prescription, casting and splint skills, so that the amount of learning could be measured quantitatively. Moreover, the last question on this questionnaire asked students about their satisfaction regarding the overall educational programs of the orthopedics course. The ratings given to each question on the questionnaire were then compared between case and control groups to determine their relationship with the application usage. Moreover, the correlation of each VAS question score with number of visits made to the corresponding part in the application within case group students were investigated.