Based on the objective, this is a fundamental study of quasi-experimental type and alternative treatment design. The study population included all students who were members of the Student Research Committee of the Arak University of Medical Sciences and were willing to participate in the study. Out of 26 health faculty students, 11 were excluded due to concurrency of unpredicted extra obligatory classes, and out of 41 medical school students, 3 were excluded due to absence from the tests. The remaining students were randomly divided into two equal groups (Table 1).
Table 1: Frequency of students based on group and major
Groups
|
Colleges
|
Medical
|
Health
|
A: Workshop / Web-based
|
(0.55)21
|
(53.3)8
|
B: Web-based / Workshop
|
17 (0.45)
|
7 (0.46.7)
|
Total
|
(0.100)38
|
(0.100)15
|
Study environment
The study was performed at Arak Medical Sciences University. The in-person workshop was held in the meeting hall, and the Internet was used for web-based education.
Instrument
The data were collected with a questionnaire including the demographic characteristics and 30 multiple-choice questions to assess the learning level of biostatistics. The instrument was a Farsi translation of a standard questionnaire called Comprehensive Assessment of Outcomes in Statistics (CAOS) as the result of the Assessment Resource Tools for Improving Statistical Thinking (ARTIST) project. This instrument is developed by Garfield and Gall in 1999 to evaluate the assessment challenges in the education of statistics and is funded by the National Science Foundation (NSF). The ARTIST website currently provides an extensive type of assessment resources for the evaluation of students’ statistical literacy (such as understanding words and signs, ability to read and interpret diagrams, and terminology), statistical reasoning (such as reasoning with statistical data), and statistical thinking (such as questioning and decision-making related to statistical data). These resources are designed to assist the faculty members and instructors teaching statistics in different majors (such as mathematics, statistics, and psychology) to assess the learning of statistics in students (23). Regarding the content of the questions, the chapters of this questionnaire include data collection and design (Chapter 1), graphical representations (Chapter 2), variability (Chapter 3), sampling variability (Chapter 4), tests of significance (Chapter 5), and bivariate data (Chapter 6).
The content validity of the statistics exam questions (CAOS) has been evaluated in two huge assessments performed in 2004 and 2006(23). The questionnaire’s items were translated into Farsi and sent to 3 experts for evaluating their content validity and consistency with the original questions: finally, 30 items were selected out of the total 40 questions.
The reliability of the original CAOS questions has been identified based on Cronbach’s alpha (0.77)(24). We used the bisection method to evaluate the Farsi version. According to the Guttman bisection scale, the final reliability of the questionnaire was 0.685, showing moderate, acceptable reliability (25). In addition, Cronbach’s alpha was calculated for all items (0.714). The test and the learners learning assessment based on different learning levels were scored according to emails with Robert delMas and Joan Garfield, the principal researchers of ARTIST (Table 2).
Table 2: Assessment method of learning levels based on items number and test number
learning levels
|
Question’s Number(Total)
|
Question’s Number(Selected)
|
statistical literacy
|
1-6-7-9-10-16-19-20-21-22-25-26-27-28-29-30-31-33-38-39
|
1-7-9-10-19-20-21-25-26-28-31-33-38
|
statistical reasoning
|
2-3-4-5-8-14-15-17-34-35-36-40
|
3-4-5-8-14-15-34-35-36-40
|
statistical thinking
|
11-12-13-18-23-24-32-37
|
11-12-13-23-24-32-37
|
The mean score of each group of items constituted a single score and was used to compare the groups in each faculty through the t-test. In addition, the paired t-test was used to evaluate changes in the learning level of each group as a pretest-posttest. Finally, the total score of the three levels of statistical literacy, statistical reasoning, and statistical thinking in the groups of each faculty was compared using the independent t-test. It should be noted that the pretest scores of the groups in each faculty were analyzed to evaluate the effect of simple random division on their equality. The reliability of other questionnaires was assessed with SPSS 16.
Inclusion and exclusion criteria
Inclusion criteria were a bachelor or higher degree student, a member of the Student Research Committee of the Arak Medical Sciences University, at least one semester passed in the relevant major, no withdrawal student, or graduated in another major than the present one. The exclusion criteria were incomplete participation in the educational courses, absence in the tests, and having the biostatistics course in the ongoing semester.
Implementation method
The total number of eligible people registered was 26 students in the health faculty and 41 students in the medical school. Out of 26 health faculty students, 11 were excluded due to concurrency of unpredicted extra classes, and out of 41 medical school students, 3 were excluded due to absence from the tests. The participants were randomly divided into two groups of 8 and 7 in the health faculty and two groups of 17 and 21 in the medical school. An educational session was held before the beginning of the study to introduce the web-based educational environment to the groups. In this session, a written consent form was also obtained from students and a pretest of all statistic questions was given. The students in both groups received an education based on the traditional (in-person) workshop learning approach (first group) and based on the web-based e-learning approach (second group) in the first session and based on web-based e-learning approach (first group) and the traditional workshop learning approach (second group) in the second session. The two sessions had a 5-day interval, and the workshop (in-person) and web-based (online) learning were held at the same time for 2 hours. Immediately after each session, an exam was given on the questions of that session. The educational content was separately prepared by two professors in the faculties. The site features, the method of presentation of subjects, the method of communication with online students, and paying attention to the sent messages were taught to professors in a private session. In total, four workshops were held in both faculties; two for the health faculty and two for the medical school. We managed the site learning environment with the OpenMeetings open-source software. To test the software abilities, we conducted a pilot educational course in two 2-hour sessions in the Payam Noor faculty in Khomein city for teaching Excel. The results of this pilot revealed that showing the professor’s image causes frequently interrupted the connection of students with the website due to weak telecommunication infrastructures; therefore, during the main study, the users only could hear the professor’s voice. In this study, we compared the sequence of blended learning using the alternative treatment design. The counterbalanced design is a method for determining the sequence of interventions in an experimental or quasi-experimental study. In the first stage of this method, with only two interventions of A and B, the researcher examines the volunteers with both interventions; and thus divides them into two groups. One group receives intervention A after intervention B, and the other group receives intervention B after intervention A. This type of counterbalanced design which consists of only two interventions of A and B is called the alternative treatment design (Figure 1) (26).
Ethical considerations
Ethical considerations consisted of stating the research objective to the officials of the Student Research Committee, the confidentiality of the characteristics and the evaluation results, honesty in all stages of the study such as from completing the questionnaire to data analysis and expression of results, and mentioning the references used in the study.