Curriculum is a dynamic process that needs continuous improvement and should be reviewed regularly. According to Stufflebeam (2004), context evaluation is defined as an assessment of the needs, opportunities, and problems that can be addressed in a particular environment [4]. The CIPP model evaluation done on the first-year integrated MBBS program showed the following results:
1. CONTEXT:
a) Educational Needs:
COVID-19 affected Pakistan in March 2020, when the session for the first-year MBBS batch 2019–20 had entered its second week. It was an emergent need to find an alternative to physical classes when all educational institutes got closed down while there was hybrid teaching for the next first-year batch, i.e., 2020–21, when restrictions on COVID-19 were relaxed and students were able to attend physical classes on campus for some time.
b) Impendent:
Due to the sudden requirement to switch to online teaching, there was a need to train the faculty. The medical education department trained the faculty on different online tools. Appropriate software was agreed upon to implement and monitor student and faculty performance. Moodle, which was already used by faculty members for some courses, was selected for asynchronous assignments for tutorials, uploading of lectures, and assessments, while webinars were purchased for lecture delivery and Zoom/Whatsapp was used for practical demonstrations and viva voce examinations. The admission of batch 2019–20 was done on PMDC criteria, in which interviews by the colleges had no weight, while batch 2020–21 was inducted during COVID-19 by PMC criteria, where 20% marks were allocated for interviews to the total marks for admission to MBBS. This challenge of admission during COVID-19 was overcome by conducting multiple mini-interviews by two faculty members for each student online. Structured scenarios on empathy, honesty, altruism, and professionalism were given to the students and graded online. Appropriate selection of online software was done with the consensus of the faculty, and monitoring was done using the recording of the lectures and sessions and giving feedback to the faculty. Coordination between the faculties was again a great challenge when everyone was doing work from home. This was facilitated by Zoom meetings by the chair for the first year. The lectures, tutorials, and practical sessions were conducted according to planners designed for online teaching. Although practical demonstrations could be shown to the students, they missed out on hands-on practice during the total number of online sessions. However, students did practice during hybrid teaching.
c) Availability of Pertinent Expertise, Assets, and Services:
The expert faculty was there to teach the students in a ratio of 3.8:1. Software for online teaching was made available, and an enabling support system was provided by the medical education department and administration.
d) Relevant Opportunities and Funding:
Appropriate funding was made available to purchase software. The spread of COVID-19 and the closure of educational institutes were so sudden that only one week was available to train the faculty for online teaching and provide them with tools, preempting the closure of them soon.
2. INPUT:
a) Approaches to meet the educational needs:
To study that the educational needs of the students during COVID-19 were met, the review of curriculum documents, admission criteria, merit list, study guides, delivery of content in the sessions, and assessments during the pandemic era were done by the first-year review committee. Admission of batch 2019–20 was done on PMDC criteria, while batch 2020–21 was done on PMC criteria, where 20% marks were given to medical colleges to interview the students. Attendance and assessment criteria were 75% and 50%, respectively, for batch 2019–20, while they were 90% and 60% for batch 2020–21, due to the change of policies by the academic council. Study guides were provided, and online planners were specially designed with software mentioned to be used. Assessments were also done online on LMS-Moodle, while viva was done on video calls.
b) Feasibility:
The purchase of hours for webinars and Zoom were done. Library and IT resources were made available online. Training of faculty was done on online teaching techniques by the medical education department.
c) Availability of Resources:
Organization of planners was done according to online sessions. Only examinable subjects were taught in the initial modules, and clinical lectures were taught in the last module. Video recording of lectures, practicals, and tutorials was done and placed on Moodle to be used by students for revision at a later time. No PBLs were conducted that year. The study plan for batch 2020–21 was partly on campus and partly online. As much experience was gained, the integration was done properly with clinical lectures side by side with the basics. PBLs were also conducted, some online and a few physically.
d) Infrastructure:
The medical school is well equipped with an auditorium, lecture halls for 150 students, tutorial/PBL rooms, practical labs that can accommodate 50 students at one time, hostels, canteens, and dining halls for 300 students at one time, a gym.
e)Human Resource:
The students' faculty ratio was 3.8:1. The faculty had highly qualified Ph.D.s in basic health sciences and foreign-qualified clinical faculty in each department. Technical staff was there to facilitate the students.
f) Administrative Support:
There were departments of student affairs, medical education, finance, IT, procurement, and marketing available to support the students.
3. PROCESS
a) Plan Implementation: The curriculum plan for the year was suddenly disturbed due to the sudden wave of pandemic COVID 19. The contingency plan faced many challenges. Faculty were trained for online teaching within a week. Software was made available. Planners were modified for e-learning. Initially, only examinable subjects were addressed as clinical faculty was facing challenges teaching clinical year students. Learning objectives as written in the guide books were covered in lectures, tutorials, and assignments on Moodle. These were analyzed by comparing the outcomes mentioned to those in the planners that were executed. The quality of lectures was good as analyzed by listening to recorded lectures by the committee members and feedback from the faculty by the students. However, variation occurs between faculty members. Practical sessions were performed by faculty and video recorded, which were uploaded to Moodle for students to see. They were not able to perform the skills. However, at the end, four weeks before mock examinations, they were called on campus to perform practicals in small groups with social distancing in place. PBLs were not conducted except for the initial two modules, which were held on campus. Tutorials were conducted via assignments. Assessments were conducted online via Moodle, with time restrictions for each. MCQs were checked by Moodle itself, and SEQs were allocated to teachers to check and place marks on Moodle, which were later tabulated. The students had a chance of cheating, and SEQs, especially, were mostly copied and pasted. Proctoring could not be done initially. The similarity index of SEQs was high amongst students. Practical test viva was taken on WhatsApp or Zoom and OSPE on Moodle. However, practicals were performed by the batch 2020–21, which attended hybrid sessions. PBLs were conducted, and assessments were also taken physically on campus. Tutorials were conducted on campus and online via Zoom in two modules. Assessments were conducted on campus except for two modules in which only MCQs were assessed online and vivas were taken on Zoom. SEQs were taken when they returned to campus. Cheating was minimized by on-campus assessments, and only MCQs and vivas were conducted online in two modules.
b) Plan Documentation:
Study guides were prepared regularly before time and distributed to the students and faculty on day one. Study guides consist of planners as well, but any changes are made via student portals. The policies and regulations book for students was also placed on the students’ portal. Attendance and assessment policies were available in the prospectus. Lecturer name and topic mentioned on planners Practicals were marked on practical books. Tutorials were documented in log books. Assessments were sent to the examination department for tabulation and item analysis. The results of individual students were uploaded and could be seen by the students through their portals. Post hoc analysis was done regularly by the examination department and analyzed by the HOD and medical education department.
c) Program Activities Scheduling:
Planners were made module-wise and placed in the study guide books regularly. Facilitator names and topics were displayed on the planners. The schedule was followed effectively. Minor changes, if they occurred, were informed to both faculty and students.
d) Program Running on Budget:
Due to COVID-19, the teaching was shifted to online, so software for classes of 150 students each for five years had to be purchased, which was over and above the planned budget for teaching and learning activities. However, as no sports activities were organized and co-curricular activities were arranged online, the budget allocated for them was not spent during COVID 19, which somehow balances the budgetary expenses.
e) Implementation Problems:
The faculty was not trained for online teaching, so it needed to be done on a war basis. One week of training was done, and then the faculty learned by themselves as they proceeded. Troubleshooting was done on a one-to-one basis by IT experts and the ME Department. Internet issues were there as students in remote areas were not able to get signals and were missing their classes. There were technical issues on part of the college as there was a crash of Moodle during assessments due to overload, and the rescheduling had to be done. Webinar failures were also frequent. Students log in for the purpose of attendance and then are attentive. There was difficulty in ensuring their engagement in lectures. Plagiarism in SEQs could not be captured due to a lack of checker facilities. The faculty, on checking the papers, reported about it. The proctoring mechanism was not available at the beginning but was later provided.
f) Quality of Process:
It was somewhat compromised, especially in the beginning when the faculty itself was getting acquainted with online techniques. However, it improved progressively as time passed and both students and faculty got used to it. Co-curricular activities, though carried out online, were not very useful in terms of the qualities required to inculcate in students. However, it gave students breathing space in difficult times. Students were lazy as they were used to studying from home. However, it took time for students to come back to their routine next year. The things progressively improved with the batch who attended in hybrid form. On campus, sports and co-curricular activities inculcate leadership qualities, sportsman spirit, team building, and working amongst students who become disciplined and punctual while attending classes physically on campus.
Feedback of students regarding faculty performance:
Students responses were calculated and % of performance on different questions asked was calculated for faculties of Basic Health Sciences department:
Students of batch 2019-20 rated the faculty of different departments as follows:
Anatomy: 82.66-92.36 Physiology: 87.23-93.79 Biochemistry: 87.95-90.26
Students found learning sessions were more student-centered with supporting online classes (p<0.05). They found that the tutorial sessions improved their problem-solving skills. Teachers were helpful in stress management during the pandemic. Proper counselors were employed by the college to cater to the needs of students who required assistance.
Students of batch 2020–21 rated the faculty of different faculties as follows:
anatomy: 78.74–88.25; physiology: 85.03–95.34; biochemistry: 86.50–92.20.
Students were satisfied with the availability of proper infrastructure, the availability of resources in the library and IT, and support from faculty and staff. They were satisfied with the environment available during teaching and assessments. Co-curricular activities are helpful in motivating them.
4.PRODUCT:
Positive and Negative outcomes:
For qualitative analysis, FGDs were carried out with 10 groups of 15 students in each group. Themes that emerged were as follows:
1. Perception of the usefulness of study guides:
Mixed opinions were obtained from Group A students, with 75% finding them helpful and 25% not finding them useful. However, group B finds them helpful, but some students suggest a need for better emphasis on how to use them (Figure 1a). A student said “Study guides are very helpful as they give us detailed learning objectives regarding the entire module”. Another quote is “it guides us which topic should be studied and how it should be done."
2. Utilization of study guidebooks:
Group A students used study guides to make short notes, summarize studies thoroughly, revise, and figure out what to study. Some students initially struggled with how to use them and were only able to do so after two modules, as commented by Group B. The comments of students were: "I know about learning objectives"; "it helps during revision as main topics are highlighted; it helps us figure out what to study without confusion and the syllabus includes everything clearly”; it should be explained in the beginning how to utilize it”.
3. Benefits of study guidebooks:
Students in Group A found study guides helpful for increasing knowledge, covering the syllabus, highlighting important topics, and filtering out significant topics. They also appreciated the learning objectives and slides prepared by the teachers. However, students in Group B found study guides helpful for defining what needs to be studied, filtering out significant topics, and leading them on a clear path (Figure 2). Students remarked: ‘We do not have to waste time wandering what to do as it helps us to study everything to the point"; “Learning objectives help to focus what to study”; It acts as a filter so that we focus on important topics first and supporting topics later on”.
4. Perception of learning outcomes:
Mixed opinions were obtained from students in group A, with some finding them well-defined and covering every topic while others did not. Students in group B found learning outcomes well-defined and covered every topic (Figure 1). The student's perception was: “Learning objectives are smart," “they are defined as what I need to know and how to learn," and “they covered each and every aspect of the topic."
5. Implementation of learning outcomes:
Students of group A found that they were not implemented in the true sense and improvements were needed, such as smaller group sessions or greater understanding of teachers related to their significance. Generally positive comments were given by students in group B, stating that most of the content was covered. No specific areas of improvement were mentioned. Students said “there should be more small group sessions for successful implementation of outcomes”; "it could be better if its significance and importance are realized”; "the syllabus was according to objectives and teachers taught accordingly”; “implementation should be improved by coordination between faculty”.
6. Effectiveness of teaching sessions:
Students generally found the teaching sessions effective and aligned with the learning outcomes, with some students suggesting the use of more multimedia and a wider spectrum of topics. However, group B students were concerned about the coordination between faculty members and found that the teaching sessions did not correspond with the learning outcomes (Figure 1). Students remarked "There should be more focus on videos and animations as they help us recall better”; "Yes, it was strictly according to learning objectives”; “teachers always showed us learning objectives at the beginning of lectures and taught us accordingly”.
7. Usefulness of SGDs and PBL:
Students generally found SGDs and PBL useful for clinically oriented knowledge, improving skills, and increasing confidence (Figure 1). However, some students found them somewhat helpful and suggested improvements such as giving topics earlier and covering a wider spectrum of topics (Figure 2 a). Students in group B found SGS and PBL useful for creating long-term memory, creating interest, and offering different perspectives. Some students suggested a need for more tutorials. Students perception was that ”PBLs help us to correlate different subjects. Scenarios are very informative and improve our knowledge”; SGDs are helpful; smaller the group, more is learning”; “SGDs help us to understand the topic and there is opportunity to ask questions more openly”; they help us to consolidate more information and could be improved by covering a wide spectrum of topics”; PBLs seem disconnected from the course”; "a different perspective on the same question is obtained and is retained for a longer time.”
8. Effectiveness of practical sessions:
Most students in Group A found practical sessions useful for improving skills, but some students found them unnecessarily long and suggested lessening the time (Figure 1). Group B students found practical sessions useful, but some students suggested allowing everyone to get the opportunity to use instruments by themselves. Students said “they are not necessarily long’; there should be more focus on performance than completing practical copies”; practicals are useful but should be done in one hour”; “they are helpful as they improve our skills”.
9. Assessment of the affective domain:
Most students found that the affective domain was mentioned in guidebooks and assessed by the faculty members. Group B students observed that the affective domain was mentioned in guidebooks but not assessed, with some students suggesting the use of log books and PBL forms for it.
10. Comparison of online teaching during COVID and in-class teaching during the non-COVID era:
Students generally found in-class teaching more effective and interactive, but they appreciated that teachers gave them slides of presentations for online teaching. Group B students found Zoom sessions useful but not webinars for online teaching during COVID.
11. Advantages and Disadvantages of Online Teaching
Recorded lectures were the most significant advantage of online teaching, as they can be accessed from home comfort and can be played again if needed. However, the lack of interaction between students and teachers and the presence of a lot of distractions were major disadvantages. The perception of group B students was that home comfort and no need to travel were the main advantages of online teaching, while network issues and a lack of practical experience were the most significant disadvantages (Figure 3b). Students remarked "In-class teaching is better as online teaching lacks interaction and students fail to focus due to limited attention span”; Usefulness depends on the type of student; if a student was exposed to technology in schooling, they would say it was better”;"Yes, to some extent, as there were some deficiencies as lack of communication and questions were tough to communicate”; “we could play lectures again”. “Zoom was useful but webinars were not”; "they were able to sleep after logging in”; home comfort”; no traveling saves time”; were not able to see bones in anatomy;network issues”.
12. Advantages and Disadvantages of On-Campus Teaching:
Students in Group A found that on-campus teaching was beneficial in terms of one-to-one interaction with teachers, more interaction with peers, and hands-on experience. However, long hours, lengthy lectures, and being time-consuming were the main drawbacks. Punctuality, routine, and interaction with teachers were advantages. In contrast, time taken for transportation and variable teacher quality were disadvantages, according to group B students. Students explained that "they can ask questions and interact with teachers”; “both were pretty much equally effective”; “more effective due to direct experience”; punctuality and routine get set”; could be able to see the bones and perform practical.”
b) Qualitative analysis of responses from administration and faculty:
The interviews conducted with the administration, which includes the principal and the Director of Student Affairs (DSA), and the faculty exposed a number of themes related to the experience of online teaching during COVID-19.
- Satisfaction with Online Teaching:
Participants had mixed feelings regarding the usefulness and satisfaction with online teaching. The principal considered it a contextual and useful option, whereas the director of student affairs (DSA) found it ineffective due to a lack of interest and two-way communication.” Forced compulsion to attend was not useful as two-way communication between teacher and students was lacking.” The faculty maintained that they had initial problems related to technical aspects, but they learned to tackle these issues in a few weeks. However, the faculty had serious concerns related to the practicals, as they could not be conducted in an online setting, hence the practical application of knowledge suffered a great deal. "Practical suffered a lot”. This concern was particularly raised by the anatomy department, as the faculty felt that gross anatomy could not be taught properly. The students, however, preferred online lectures as they did not have to travel or commute, so they could concentrate more on their studies. Challenges Faced During the COVID-19 era, the administrators and faculty faced various challenges in teaching and assessment. The principal expressed concerns regarding the inability of senior faculty members to operate online modalities and utilize them appropriately. All the respondents unanimously agreed that network issues were a major hindrance to conducting online classes smoothly (Figure 3). The faculty also stated that proctoring during assessments was ineffective and students could easily cheat. "Copy and paste was observed in SEQs, particularly.” According to the faculty, they were only able to demonstrate the skill, but they lacked a means of determining whether or not the students had actually acquired the skill. Similarly, problem-based learning (PBL) sessions could not be conducted, leading to unsatisfactory results. The DSA noticed that even after the lockdown had lifted, fewer students were attending the classes. They blamed the lack of engagement during online lectures for this fallout in face-to-face lecture attendance, as students were finding it hard to return to the engaging routine of lectures post-COVID. ”Students were not coming back to routine for face-to-face sessions after COVID-19 restrictions were lifted.”
2) Faculty Training and Performance:
The participants had mixed views regarding the faculty's training and performance. The principal suggested that reverse mentoring might help in troubleshooting technology issues with senior faculty. "Junior faculty is more tech savvy, so reverse mentoring helped a lot during COVID-19. The DSA considered the faculty quick in catching up with technology, and the faculty indicated that the Medical Education department trained them well in time.” "The Medical Education department was very supportive.” The administration claimed that students mostly had positive feedback regarding faculty’s performance because regardless of the quality of teaching, the students were happy to stay at home and take lectures (figure 2). Differences between Online and On-Campus Teaching The participants identified various differences between online and on-campus teaching. Physical presence, eye contact, and gestures were missing in online teaching, and non-verbal communication was lacking, leading to less effective teaching. However, the faculty believed that blended learning could be utilized post-COVID.” Students lack eye contact and physical presence of teachers”;” Teacher control of class was missing.”
3.Funding for Online Teaching:
The participants agreed that not much funding was required for implementing online teaching during the COVID-19 pandemic. The principal mentioned that the savings from electricity and transportation balanced the funding requirements. However, the DSA suggested that funding was required for cameras, Zoom, webinars, and laptop devices. "The medical education department was not equipped initially. Had to purchase webinars and Zoom hours.” Satisfaction with Study Guides and Planners Overall, the participants were satisfied with the faculty's job in a short time to take over as compared to other institutes. ”Variation in teaching quality was reported by the students.”
4)Medical Education's Preparedness for Online Teaching :
The participants had contrasting viewpoints regarding Medical Education's preparedness for online teaching. The principal suggested that teaching technologies should be made more available. The DSA stated that the department of medical education focused more on faculty training and suggested that student training should also be emphasized. The faculty mentioned that they were effectively trained by the department of medical education but added that there is always room for improvement (Figure 2b) in the admission process during COVID-19. The participants agreed that the admission process was not much of a challenge. Multiple mini-interviews (MMI) were successfully conducted online, and more students were interviewed in a short span. However, an assessment of non-verbal communication was lacking. ”MMI makes the process easy”; “MMI was an excellent mode of interviewing online.”
5.Progress Monitoring of Online Teaching:
According to the participants, progress monitoring of online teaching was carried out based on feedback from both students and faculty, while any technological issues were handled by experts.
6.Messages for Students and Faculty:
The participants had various messages for students and faculty. The principal suggested that technology is here to stay, and we have to develop these skills to survive. "Technology has to stay, so better get tech-savvy”. DSA suggested that faculty should focus on delivery, and students should try to be good doctors and better serve humanity. The faculty suggested that students and faculty should be well-prepared for online teaching, and the medical education department should conduct workshops on online teaching and learning at least twice annually (Figure 3).
Intended Outcomes:
The two batches were assessed in module and professional examinations, which were conducted online and on campus. The merit of admission was evaluated for the two batches and was correlated to the academic performance of the batches. The Cronbach's alpha, difficulty, and discriminatory index of the papers prepared for module examinations by the faculty of basic sciences were also assessed. The quality of delivery by the faculty was also gauged by listening to the recorded YouTube lectures. It was found that although the merit of admission for batch 2020–21 was less than that of class 2019–20, the performance of batch 2020–21, which was taught in hybrid form (partly online and partly on campus), was significantly better in modules and professional examinations in three basic subjects, i.e., anatomy, physiology, and biochemistry, compared to batch 2019–20, which was taught totally online. The quality of papers and teaching was equally good online and on campus, but lack of attention by the students sitting at home, low motivation and mental stress of the pandemic effecting their near and dear ones and themselves with ineffective proctoring mechanisms, and being unable to have the physical presence of a teacher in class resulted in low performance of the students for the batch of totally online teaching compared to hybrid teaching. Lack of co-curricular activities also added to it.
Un-intentioned Outcomes:
Online teaching during the pandemic resulted in effective teacher training for blended sessions and courses. The online courses could be designed and given to students as supplementary help. Moreover, students also get accustomed to online teaching and assessment, and low-stakes examinations can be conducted online on LMS (Moodle), which saves time and money at both ends.
SHORT-TERM IMPLICATIONS: The batch exposed to only online teaching during the pandemic realized the importance of attending the medical school, which not only imparts quality education but also grooms the students in co-curricular activities, problem solving, team building, leadership skills, and counseling them whenever required. Moreover, faculty realized the importance of being tech-savvy and the significance of blended learning, which can make students more responsible for their studies. The need for strengthening the medical education department in terms of online teaching and training of faculty regularly was identified. The 2019–20 batch scored 92.58% passing rate in the university professional examination, while the 2020–21 batch scored 98.16%. The professional papers in basic sciences for batch 2019–120 were easy compared to the other batch as evaluated by faculty involved in teaching and assessment. LONG-TERM IMPLICATIONS:
The online program is an effective alternative to on-campus teaching in blended form. Both batches performed better in the next two years and gave 97-98% results in 2021 and 2022. However, long-term results will be obtained when these batches graduate and work in hospitals.
IMPACT OF THE PROGRAM :
The online program was successful in hybrid (blended form) with certain limitations for total online, which are highlighted in the results.
EFFECTIVITY OF THE PROGRAM:
Most of the students were able to perform well, especially in hybrid learning methods, which emphasize the need for blended learning both in asynchronous and synchronous forms.
SUSTAINABILITY OF THE PROGRAM:
The program is built into the LMS systems; however, webinars and Zoom need to be purchased whenever needed. This does not have a significant financial burden on the institute, so it is sustainable in its current or improved form.
HOW EASY CAN THE PROGRAM BE ADOPTED BY OTHERS IN NEED?
The program is easily available and cost-effective. However, effective training of faculty and students with proper support from administration and medical education makes it a cost-effective, sustainable model that can be easily adopted by other institutes. Quantitative Results:
Table 1 shows an analysis of module examinations prepared during total online and hybrid sessions. Cronbach’s alpha of the papers during hybrid sessions for the basic subjects was better than the online sessions. However, difficulty and discriminatory indexes were comparable.
Table 2 shows the comparison of admission scores, internal assessment scores, and professional examination scores between first-year MBBS (2019-2020-online) and 2020-2021 hybrid batches. The admission merit, particularly in terms of MDCAT, and the final merit of the online batch were significantly higher (P< 0.01) compared to the batch of 2020–21, which was taught in a hybrid fashion. The internal assessments of anatomy improved significantly (P<0.01) in hybrid teaching compared to online teaching, while physiology and biochemistry remained comparable in two batches. The first professional results of anatomy and physiology in the hybrid teaching batch improved significantly (P< 0.01), while biochemistry results remain comparative in two batches.
Table 3 shows that admission criteria for batch 2019–20 (online) were based on PMDC criteria, in which MDCAT results were considered along with matriculation and FSC results, while the 2020–21 (hybrid) batch was inducted based on PMC criteria, where 20% marks were given to medical colleges to conduct interviews. Multiple mini-interviews were conducted by the admissions cell of SMDC for admission. The internal policy of SMDC was 70% attendance and 50% passing criteria to be eligible to take the professional examination for batch 20219-20, but 90% attendance and 60% assessment for batch 0f 2020-21. Overall, University first professional result was 92.67%, with 11 students failing and 5 being declared ineligible to take exams based on attendance and assessment criteria for the 2019–20 (online) batch, while it was 98.58% with 2 failing and 11 ineligible students for the 2020–21 (hybrid) batch.
Table 1: Analysis of Module Assessments Reports based on Difficulty and Discriminatory index and Cronbach’s alpha
|
|
|
Anatomy 2019-2020
|
Anatomy 2020-2021
|
Physiology 2019-2020
|
Physiology 2020-2021
|
Biochemistry 2019-2020
|
Biochemistry 2020-2021
|
|
Test 1
|
Cronbach’s alpha
|
0.49
|
0.8
|
0.25
|
0.71
|
0.24
|
0.73
|
|
|
Difficulty Index
|
3
|
6
|
5
|
1
|
4
|
2
|
|
|
Discriminatory index
|
2
|
1
|
6
|
1
|
3
|
2
|
|
Test 2
|
Cronbach’s alpha
|
0.55
|
0.73
|
0.1
|
0.56
|
0.71
|
0.51
|
|
|
Difficulty Index
|
3
|
4
|
1
|
4
|
1
|
3
|
|
|
Discriminatory index
|
1
|
1
|
1
|
1
|
2
|
0
|
|
Test 3
|
Cronbach’s alpha
|
0.69
|
0.45
|
0.78
|
0.77
|
0.68
|
0.46
|
|
|
Difficulty Index
|
2
|
4
|
3
|
3
|
2
|
6
|
|
|
Discriminatory index
|
0
|
1
|
1
|
1
|
2
|
2
|
|
Test 4
|
Cronbach’s alpha
|
0.73
|
0.61
|
-
|
064
|
-
|
0.76
|
|
|
Difficulty Index
|
5
|
1
|
-
|
3
|
-
|
4
|
|
|
Discriminatory index
|
2
|
2
|
|
1
|
|
4
|
|
Test 5
|
Cronbach’s alpha
|
0.57
|
0.64
|
-
|
-
|
-
|
-
|
|
|
Difficulty Index
|
3
|
6
|
-
|
-
|
-
|
-
|
|
|
Discriminatory index
|
2
|
1
|
-
|
-
|
-
|
-
|
|
Test 6
|
Cronbach’s alpha
|
0.69
|
0.68
|
-
|
-
|
-
|
-
|
|
|
Difficulty Index
|
12
|
6
|
-
|
-
|
-
|
-
|
|
|
Discriminatory index
|
3
|
0
|
-
|
-
|
-
|
-
|
|
Test 7
|
Cronbach’s alpha
|
-
|
0.69
|
-
|
-
|
-
|
-
|
|
|
Difficulty Index
|
-
|
3
|
-
|
-
|
-
|
-
|
|
|
Discriminatory index
|
|
3
|
-
|
-
|
-
|
-
|
*Number shown against the difficulty and discriminatory index are the number of faulty items identified in each examination.
Table shows reliability of assessments were low in the beginning of COVID- 19 Pandemic i.e. in 2019-2020 (totally online) but improved with passage of time in the three basic sciences subjects in 2020-21 (hybrid)
Table 2: Mean differences between admission scores, internal assessment scores and first professional scores between SMDC MBBS students of years 2019-2020 (online) and 2020-21 (Hybrid).
|
Characteristic (n for 2019-20 | n for 2020-21)
|
2019-20
|
2020-21
|
P-value*
|
|
Admission (143 | 150)
|
MDCAT
|
88.87 ± 2.37
|
85.76 ± 3.05
|
< 0.01*
|
|
F.Sc
|
89.81 ± 2.10
|
89.84 ± 2.95
|
0.92
|
|
Final merit
|
89.33 ± 1.29
|
85.13 ± 1.65
|
< 0.01*
|
|
Internal assessment (146 | 154)
|
Anatomy
|
57.28 ± 8.55
|
60.36 ± 8.48
|
< 0.01*
|
|
Physiology
|
67.38± 11.11
|
64.55± 11.51
|
0.03*
|
|
Biochemistry
|
64.79± 10.43
|
67.70± 11.69
|
0.02*
|
|
Professional examination (148 | 150)
|
Anatomy
|
67.83 ± 7.32
|
70.63 ± 6.80
|
< 0.01*
|
|
Physiology
|
71.23 ± 6.71
|
75.11 ± 5.41
|
< 0.01*
|
|
Biochemistry
|
75.90 ± 7.22
|
76.30 ± 6.33
|
0.61
|
*ascertained by Independent samples’t-test
Table 3: Comparison of Admission criteria, Attendance Assessment Policy and Performance of students in Modular and professional Examinations (Online versus Hybrid Teaching)
|
|
Batch 2019-20
Online
|
Batch 2020-21
Hybrid
|
P value
|
|
Admission criteria
|
PMDC Criteria
No MMI conducted
|
PMC Criteria
MMI conducted with 20% marks
|
|
|
Attendance policy SMDC
|
75%
|
90%
|
<0.01
|
|
Assessment Policy
SMDC
|
50%
|
60%
|
<0.01
|
|
UHS First Professional Result (overall)
|
92.67%
|
98.58%
|
<0.05*
|
|
No of students
Who failed annual prof exam
|
11
|
2
|
< 0.01*
|
|
No of ineligible students
|
5
|
11
|
< 0.05*
|
|
No of students who appeared in supplementary examinations
|
16
|
13
|
>0.05
|
|
Students failed in second annual supplementary examination
|
3
|
-
|
>0.05
|
*P value significant