The results of this research work are presented in two sections: reliability and validity studies of the survey and analysis of the responses of study participants.
Section I: Reliability and validity studies of the survey:
The reliability study revealed an excellent internal consistency of the survey (Cronbach’s alpha = 0.925).
The construct validity of the survey was established through EFA (Table 1 and Figure 1).
Table 1: Exploratory Factor Analysis (EFA) of the survey:
No.*
|
Item
|
Factor 1‡
|
h2
|
11
|
Overall, formative assessment of practical anatomy has been helpful to my learning experience
|
0.84
|
0.70
|
2
|
Formative assessments helped me identify my misunderstandings in human anatomy
|
0.81
|
0.66
|
6
|
Formative assessments helped me identify easily forgettable important details in human anatomy
|
0.80
|
0.64
|
9
|
Formative assessments helped me prepare for the summative OSPE
|
0.79
|
0.62
|
10
|
I expect formative assessments to reduce my OSPE exam-related anxiety
|
0.78
|
0.62
|
3
|
Formative assessments helped me improve my understanding of human anatomy
|
0.78
|
0.60
|
7
|
Feedback given by the instructors after formative assessments was helpful to my learning experience
|
0.76
|
0.57
|
5
|
Formative assessments helped me evaluate my knowledge level in human anatomy
|
0.75
|
0.57
|
8
|
I utilize the feedback from formative assessments to guide my studying for future exams
|
0.75
|
0.57
|
4
|
I feel confident in my knowledge of human anatomy after completing the formative assessments
|
0.73
|
0.53
|
12
|
I wish to have formative assessment in upcoming educational units
|
0.69
|
0.48
|
1
|
Formative assessments in human anatomy match well with the content covered in theoretical and practical sessions
|
0.53
|
0.28
|
Variance (%)
|
56.9%
|
|
Eigenvalue
|
6.8
|
|
*Total final items are 12 items. No items were deleted. Items are sorted from the item with the highest loading to the item with the lowest loading.
‡Factor label is Factor 1: Student Perception (n=12 items).
Abbreviation: h2 = item communalities
Table 1 shows the results of the exploratory factor analysis. The table shows that all items of the survey have loadings on one factor, with item 11 having the highest loading of 0.84. The communalities for all items range from 0.28 to 0.70, indicating that they are well-represented by the factor and may be more reliable. The eigenvalue for the one factor is 6.8, which explains 56.9% of the variance in the data. The overall fit of the model is good, as indicated by a Kaiser-Meyer-Olkin measure of 0.94 and a statistically significant Bartlett's test of sphericity (c2 = 1169.02, p=0.000). Based on these results, it appears that the factor represents a strong underlying construct, which could be conceptualized as “Student Perception”.
Figure 1 shows the scree plot of the EFA. The scree plot shows a clear break at the first factor, indicating that only one factor should be retained. The slope before the scree point is steep, suggesting that the first factor explains the largest portion of the variance in the data.
Section II: Analysis of study participants’ responses:
The response rate was 80.5% (n=173). Female students constituted about two-thirds of the study sample. Data was collected through a survey prepared by the authors to explore their perception of the formative assessment and feedback introduced at the anatomy practical sessions. Data on student’s performance in final anatomy practical exams was obtained from the assessment documents after taking the necessary permissions from the CMMS-AGU administration.
Table 2: Comparison of mean marks in Unit 2 (without formative assessment) and Unit 3 (with formative assessment) in the academic year 2023-2024 using paired samples t-test:
Unit
|
n
|
Mean Mark (±SD)
|
Min – Max
(Top mark = 20)
|
t
|
Sig. (p-value)
|
Unit 2 (RS)
(No formative assessment and feedback)
|
173
|
11.62 (±4.55)
|
1 – 20
|
12.04
|
0.0001*
|
Unit 3 (CVS)
(With formative assessment and feedback)
|
173
|
14.22 (±4.64)
|
4 – 20
|
* Statistically significant
Table 2 indicates that the mean mark for Unit 3 (14.22) (wherein formative assessment was conducted) is higher than that for Unit 2 (11.62) (wherein no formative assessment was conducted). The difference is statistically significant (t = 12.04; p = 0.0001). The findings in the table suggest that the formative assessment may have had a significantly positive effect on students' performance.
Table 3: Comparison of mean marks in Unit 3 in the academic year 2023-2024 (with formative assessment) and the academic year 2022-2023 (without formative assessment) using independent samples t-test:
Unit
|
n
|
Mean Mark (±SD)
|
Min – Max
(Top mark = 20)
|
t
|
Sig. (p-value)
|
Unit 3 (CVS; 2022-2023)⁑
(No formative assessment and feedback)
|
168
|
12.59 (±5.22)
|
2 – 20
|
3.10
|
0.002*
|
Unit 3 (CVS; 2023-2024)
(With formative assessment and feedback)
|
173
|
14.22 (±4.64)
|
4 – 20
|
* Statistically significant
⁑ Marks of the students of the previous academic year were used as a control group for comparison.
Table 3 indicates that in the academic year 2023-2024, where formative assessment and feedback were implemented, the average mark in Unit 3 was higher compared to the previous academic year 2022-2023, where formative assessment and feedback were not implemented. The difference in mean marks between the two groups was found to be statistically significant (t = 3.10; p = 0.002). This suggests that the introduction of formative assessment and feedback had a positive impact on the students' performance in Unit 3.
Table 4: Means and standard deviations of students’ responses regarding to their perception of formative assessment and feedback (n=173):
No.
|
Item
|
Mean (±SD)
|
Min – Max
|
1
|
Formative assessments in human anatomy match well with the content covered in theoretical and practical sessions
|
4.53 (±0.70)
|
3 – 5
|
2
|
Formative assessments helped me identify my misunderstandings in human anatomy
|
4.58 (±0.67)
|
1 – 5
|
3
|
Formative assessments helped me improve my understanding of human anatomy
|
4.51 (±0.75)
|
1 – 5
|
4
|
I feel confident in my knowledge of human anatomy after completing the formative assessments
|
4.35 (±0.78)
|
1 – 5
|
5
|
Formative assessments helped me evaluate my knowledge level in human anatomy
|
4.54 (±0.70)
|
1 – 5
|
6
|
Formative assessments helped me identify easily forgettable important details in human anatomy
|
4.48 (±0.75)
|
1 – 5
|
7
|
Feedback given by the instructors after formative assessments was helpful to my learning experience
|
4.50 (±0.73)
|
1 – 5
|
8
|
I utilize the feedback from formative assessments to guide my studying for future exams
|
4.47 (±0.74)
|
1 – 5
|
9
|
Formative assessments helped me prepare for the summative OSPE
|
4.42 (±0.81)
|
1 – 5
|
10
|
I expect formative assessments to reduce my OSPE exam-related anxiety
|
4.36 (±0.86)
|
1 – 5
|
11
|
Overall, formative assessment of practical anatomy has been helpful to my learning experience
|
4.61 (±0.60)
|
1 – 5
|
12
|
I wish to have formative assessment in upcoming educational units
|
4.58 (±0.79)
|
1 – 5
|
Table 4 presents the means and standard deviations of students' responses to a survey regarding their perception of formative assessment and feedback conducted for human anatomy demonstration sessions. The mean scores ranged from 4.35 to 4.61 out of 5. Item 11 has the highest mean score, indicating that students found formative assessment of practical anatomy to be particularly helpful to their learning experience. Item 4 has the lowest mean score, indicating that students were relatively less confident in their knowledge of human anatomy after completing the formative assessments. Generally, the results suggest that students have a positive perception of formative assessment and feedback conducted during the sessions.
Table 5: Comparison between males and females regarding their perception of formative assessment and feedback:
No.
|
Item
|
Males (n=52)
|
Females (n=121)
|
Sig. (p-value)
|
Mean (±SD)
|
Mean (±SD)
|
1
|
Formative assessments in human anatomy match well with the content covered in theoretical and practical sessions
|
4.60 (±0.60)
|
4.50 (±0.73)
|
0.426
|
2
|
Formative assessments helped me identify my misunderstandings in human anatomy
|
4.54 (±0.64)
|
4.60 (±0.68)
|
0.558
|
3
|
Formative assessments helped me improve my understanding of human anatomy
|
4.42 (±0.78)
|
4.55 (±0.74)
|
0.328
|
4
|
I feel confident in my knowledge of human anatomy after completing the formative assessments
|
4.33 (±0.76)
|
4.36 (±0.78)
|
0.825
|
5
|
Formative assessments helped me evaluate my knowledge level in human anatomy
|
4.52 (±0.67)
|
4.55 (±0.72)
|
0.823
|
6
|
Formative assessments helped me identify easily forgettable important details in human anatomy
|
4.46 (±0.73)
|
4.46 (±0.77)
|
0.835
|
7
|
Feedback given by the instructors after formative assessments was helpful to my learning experience
|
4.58 (±0.67)
|
4.47 (±0.75)
|
0.382
|
8
|
I utilize the feedback from formative assessments to guide my studying for future exams
|
4.44 (±0.75)
|
4.49 (±0.73)
|
0.712
|
9
|
Formative assessments helped me prepare for the summative OSPE
|
4.44 (±0.75)
|
4.41 (±0.83)
|
0.829
|
10
|
I expect formative assessments to reduce my OSPE exam-related anxiety
|
4.40 (±0.85)
|
4.35 (±0.86)
|
0.691
|
11
|
Overall, formative assessment of practical anatomy has been helpful to my learning experience
|
4.65 (±0.56)
|
4.60 (±0.61)
|
0.553
|
12
|
I wish to have formative assessment in upcoming educational units
|
4.52 (±0.85)
|
4.60 (±0.77)
|
0.524
|
Table 5 presents a comparison between males and females in their perception of formative assessment and feedback in conducted for human anatomy demonstration sessions. The mean scores for males and females are generally similar, with only small differences observed in some items. For example, males have a slightly higher mean score in item 1 ("Formative assessments in human anatomy match well with the content covered in theoretical and practical sessions"), while females have a slightly higher mean score in item 3 ("Formative assessments helped me improve my understanding of human anatomy"). Overall, the results show that there are no statistically significant differences in the perception of formative assessment and feedback between males and females (p > 0.05).
Table 6: Frequencies and percentages of students’ responses to open-ended questions of the survey including quotes by students:
Question
|
Response Themes
|
Response Frequency
|
Response Percentage
|
Quotes by Students
|
What are the main benefits you gained from formative assessment and feedback sessions?
|
Preparing me for the final end-of-unit OSPE exam
|
46
|
43.4%
|
“I consider the formative assessment and feedback sessions as mock exam that trained me on dealing with the OSPE questions at the end-of-unit exam.”
“Attending formative assessment sessions gave me deeper understanding of the main anatomical concepts, especially because of the feedback I received from the tutors.”
“I am happy this assessment reduced my anxiety in the final exam as I now know how the questions come.”
|
Understanding the subject of anatomy and remembering the easily forgettable information
|
29
|
27.4%
|
Assessing my knowledge in human anatomy and knowing my weaknesses
|
17
|
16%
|
Reviewing what I learned in the practical sessions
|
9
|
8.5%
|
Reducing the stress and anxiety related to the summative practical exam
|
5
|
4.7%
|
Total Responses
|
106
|
100%
|
What are your suggestions to improve formative assessments and feedback sessions?
|
Uploading questions with explanatory feedback on the learning management system (Moodle) for self-review
|
20
|
43.5%
|
“I think it would be useful if we can have a larger number of review questions on Moodle [LMS] so we can revisit all the time from our homes."
"Continue this initiative, but with more questions that cover not only gross anatomy and histology but also embryology.”
|
Adding more questions to the formative exam
|
19
|
41.3%
|
Diversify the types and scopes of questions to also include embryology
|
4
|
8.7%
|
Conducting a formative quiz after theoretical teaching session
|
3
|
6.5%
|
Total Responses
|
46
|
100%
|
Table 6 presents the students' responses to the open-ended questions about the benefits they gained from the formative assessment and feedback sessions, as well as their suggestions to improve these sessions.
Regarding the benefits of the formative assessment and feedback sessions, the most mentioned benefit, reported by 43.4% of the students, was that the sessions helped them prepare for the final end-of-unit practical exam. Other benefits mentioned by the students included understanding the subject of anatomy and remembering easily forgettable information (27.4%), assessing their knowledge in human anatomy, and identifying their weaknesses (16%), and reviewing what they learned in the practical sessions (8.5%). A smaller percentage of students reported that the sessions helped reduce stress and anxiety related to the summative practical exam (4.7%).
In terms of suggestions to improve the formative assessments and feedback sessions, the most frequently mentioned suggestion, by 43.5% of the students, was to upload questions with explanatory feedback on Moodle for self-review. Other suggestions included adding more questions to the formative exam (41.3%), diversifying the types and scopes of questions to include embryology (8.7%), and conducting a formative quiz after each teaching session (6.5%).