The results showed that students felt that lengthy exam content or a large number of questions resulted in wasting time, particularly when the exam is in English. The literatures indicated that English speakers could read up to 150 words/min (22, 23). However, in the case of non-native English speakers, the reading time for exams was expected to be longer. Furthermore, exam reading was focused on decisive reading rather than comprehensive reading, where the main goal was to make informed decisions. Therefore, using the English reading rate as a basis for determining exam duration was not suitable within the context of Thai students, as English was not their daily language of communication.
In the field of medical education in Thailand, the MCQ exams in Medicine often refer to criteria set by the Thai National Licensing Examination Test, which recommends a duration of 60 minutes for 50 questions (12, 43). Alternatively, the National Board of Medical Examiners (NBME) recommended answering each question within 60–90 seconds (17, 40). However, there may be limitations if the exam questions contain lengthy content that cannot be comprehended and decided upon within one minute. Furthermore, the difficulty level of the exam questions is often established as a passing criterion, prioritizing validity considerations in terms of content format and achieving the intended objectives. Moreover, students naturally desire to obtain the highest possible score on the exam, regardless of the level of difficulty or length of the exam. Therefore, it is important for students to manage their time effectively to ensure they can complete all the exam questions within the given timeframe (11, 35, 36)
The qualitative results indicated that regardless of the exam duration set by the administering professor, students ultimately would complete the exam within the allotted time frame. This often involves guessing answers towards the end of the exam period, as students may not have enough time to complete the exam thoroughly. It was observed that students tend to guess exam questions towards the end of the exam period, approximately 5 minutes before time runs out. Studying the context of examination time from the students' perspective is crucial to determine an appropriate exam format. This enables the reduction of threats to validity by minimizing students’ guessing due to time constraints during the exam (11)
Additionally, students agree that application and calculation questions on the exam require more time to read and decide upon (50), as opposed to questions with figures and tables that aid in faster decision making. Based on these findings, it can be concluded that comprehensive reading rates may not be a reliable indicator of decision-making reading rates, particularly in the context of medical school exams. Therefore, studying decision-making reading rates within the context of medical school exams is crucial.
Based on the data on the distribution of exam items according to the taxonomy, specifically Recall (reducing decision time for selecting answers) - Comprehension (neutral) - Application/Calculation (increasing decision time for selecting answers), it can help determine whether the exam time should be increased appropriately. Student feedback also indicates that if an exam set has a significant proportion of application/calculation items, exceeding one-third of the total number of questions, there is a tendency to not complete the exam in time and resort to guessing answers before time runs out.
The researcher, therefore, examined the domain and specific factors related to the characteristics of the MCQ test. In addition, the researcher limited the study scope to only English tests given to non-native English speakers, and onsite computer-based tests, which removed unrelated factors that may affect the duration of the exam, and the results of the analysis appeared as follows.
First, the factors that tend to be a positive correlation with the duration of the exam (negative correlation and decision making) were the number of exams, number of words, calculation exam, and analytical thinking exam.
Second, the factors that tend to be a negative correlation with the duration of the exam (positive correlation and decision making) were recall questions as well as questions with given images, and tables.
The factor that led to longer reading times for decision-making purposes was when the exam had a greater proportion of application or calculation questions, comprising over 33% of the exam questions (as expressed in the students' qualitative data). Therefore, it is not recommended to analyze the exam completion time based on data obtained from the analysis of reading comprehension since it is for decision-making purposes. Furthermore, it should be noted that these factors constitute internal threats to validity (ref), but they can be controlled to ensure that examination tools can be used effectively and aligned with the intended objectives. When incorporating the data obtained from research, it can lead to the identification of new themes related to factors influencing examination time.
Five constructive domains included: 1) the number and total word count, 2) positive factors of difficulty (application/calculation questions), 3) negative factors of difficulty (recall question), 4) examiners (non-native English speakers or not) and 5) pictures/symbols in tests.
In conjunction with the examination-related factors, students also acknowledge the importance of considering the test environment within the examination room. This is crucial for promoting student concentration and facilitating accurate response selection aligned with the objectives of the assessment tool. It aligns with the existing literature, which indicates that the test environment constitutes a construct irrelevant threat to the validity of educational measurement (35–37).