Background of the study
Artificial Intelligence (AI) is the technology that seeks to build systems that mimic the intelligence of humans or systems to perform tasks that require the intelligence of humans (Russell, 2010). The development of AI is growing fast due to the availability of vast amounts of data, making it easy to train machines to learn and make decisions (Duan et al., 2019). The exponential growth of computing power also characterises the growth in AI; thus, developing specialised hardware such as graphics processing units (GPUs) and tensor processing units (TPUs) has accelerated AI training and inference process (Khan et al., 2021; Cowls et al., 2021; Pietrzak et al., 2023). Today, AI is emerging as a transformative force with its applications spanning various industries, such as healthcare and medical surgery (Hashimoto et al., 2020; Gamble, 2020). In the financial sector, machine learning algorithms detect fraud and predict stocks (Omar et al., 2017; Rath et al., 2019). In the transportation sector, AI is transforming the driving experience with the advent of self-driving cars (Zhao et al., 2018). The influence of AI is also felt in the areas of construction, sports, entertainment, and education. For instance, AI implementation positively impacts construction safety (Dobrucali et al., 2022). In sports, augmented reality and data analytics have transformed sporting activities' viewing and reporting experience. Entertainment has become much more fun than ever as AI has been incorporated into media streaming platforms where users have a personalised viewing and listening experience (Morris & Patterson, 2015).
Equally, it is worth noting that the educational sector has also experienced a significant transformation due to the substantial impact of AI (Alam, 2021; Gong et al., 2021). Several studies have contributed to knowledge by exploring the implications of AI use in education. Such areas of education include Early Childhood Education, Primary Education, Online Learning and Higher Education. Findings from Su et al. (2023) indicate that AI learning in Early Childhood Education presents opportunities for fostering young children's AI literacy; it can also facilitate the understanding of AI concepts, practices, and perspectives at an early stage. (Pardamean et al., 2022), with their study in Indonesia, found that using AI in online learning can satisfy students’ learning styles at the primary level of education. Studies by (Dai et al., 2023; Ng et al., 2022) have all sought to highlight the importance of AI in Early Childhood Education. At the Higher Education level, AI has proven to be successful in impacting students’ learning, teaching, assessment, and administrative tasks (Al Braiki et al., 2020; Lameras & Arnab, 2021; Ahmad et al., 2022; Xia et al., 2022; Pham & Sampson, 2022; Fiok et al., 2022; Hannan & Liu, 2023).
What is ChatGPT, and how does it work?
The AI tool that has currently taken centre stage is ChatGPT, an AI chatbot that was launched in November 2022 by OpenAI. ChatGPT is a chatbot built on the GPT (Generative Pre-Trained Transformer) language model; it provides an interactive environment where users can converse with the Chatbot more naturally (OpenAI, 2022). This chatbot gained 100 million subscribers two months after its launch (Milmo, 2023). Users interact with ChatGPT through ‘prompting’, the technical term for querying the chatbot to engage it. Generative Pre-trained Transformer (GPT) is the name given to a family of natural language models developed by OpenAI. The language model on which ChatGPT is built utilises the technique of reinforcement learning with human feedback. Reinforcement learning is a subfield of machine learning techniques where an agent learns to take actions in an environment in a way that maximises rewards (Devi et al., 2023). The agents learn without direct supervision and must determine which actions yield the highest rewards. The rewards that act as feedback reinforce the successful actions of the agent, thus contributing towards the agent’s goal (Gilbert et al., 2023). GPT is part of the series of language models built by OpenAI. The GPT language model is a type of large language model (LLM) trained by leveraging neural networks and an enormous amount of data, mainly from the Internet, information licensed from third parties and information provided as feedback from users and human trainers (Kublik & Saboo, 2022).
The iconic feature of ChatGPT has to do with its generative AI capabilities, thus its ability to produce original results (Qiu, 2023). The chatbot’s ability to generate original content and provide relevant information to the user’s prompt makes it a good tool for teachers and learners (Kohnke et al., 2023). ChatGPT’s ability spans various functions like performing sentiment analysis on texts, performing data analysis, writing articles, proofreading texts, and providing coding guides to students (Koubaa et al., 2023a).
Users, by way of prompting, can direct the chatbot to assume a role and perform a task. For example, a student can task ChatGPT to assume the role of a teacher in a subject area to teach and guide the student to learn a programming language. ChatGPT, with its generative AI capabilities, possesses the potential to impact how we generate content and engage with content. Additionally, it can potentially revolutionise the creative arts industries and the communication media (Korzynski et al., 2023; Qiu, 2023; Ritala et al., 2023).
Additionally, the disruptive nature of ChatGPT has the potential to revolutionise teaching, learning and research as revealed by Ivanov and Soliman (2023a). In their concept paper, the researchers argue that, with ChatGPT's capabilities to generate original content, students can use this to write their assignments and research papers. This will mean that Universities will be forced to reevaluate their teaching and assessment strategies and incorporate generative language models in teaching. They further argue that publishers must be more receptive towards manuscripts that AI partially generates.
Capabilities of ChatGPT
Studies have begun to explore the applications and implications of ChatGPT in Higher Education. For example, a study byTalan and Kalinkara (2023), compared ChatGPT's performance to 37 undergraduate students on an anatomy exam. ChatGPT answered 27 out of 40 questions correctly (67.5%), outperforming students who averaged 21.1 correct answers (52.8%). The study's findings suggest ChatGPT has potential as an educational tool but caution against risks like cheating. They stress the need for further research to understand ChatGPT's educational implications. Other studies have focused on ChatGPT's potential as an academic writing assistant, with mixed findings on its ability to produce high-quality texts without guidance (Cardon et al., 2023; Imran & Almusharraf, 2023). This study aimed to build on these works by examining the potential of ChatGPT specifically for aiding in examination writing in Higher Education contexts. A summary of the ChatGPT’s usage and capabilities is shown in the diagram (Fig. 1) below:
Limitations of ChatGPT
Despite the amazing capabilities of ChatGPT, it is not without some limitations; the chatbot can sometimes produce inappropriate responses. It can also hallucinate; this is a situation where the chatbot generates responses that may be plausible but factually incorrect or unsupported by the knowledge and data the system has been trained on. This phenomenon results from reliance on predictive modelling and probability rather than fully understanding concepts or having complete knowledge of the context of the prompt (Koubaa et al., 2023b; Roumeliotis & Tselikas, 2023; Sok & Heng, 2023).
Ethical concerns with the use of ChatGPT amongst students
Policymakers and educators have always grappled with the implications of incorporating AI tools into learning environments (Pedro et al., 2019; Roschelle et al., 2020; Miao et al., 2021). What ChatGPT presents will not be an exception; at the same time, ChatGPT stands out as an excellent tool for teachers and students; its fast adoption and use in educational settings, primarily by students, has raised some ethical concerns. The use of chatbots has sparked debates on its consequences on academic integrity and, most significantly, the development of critical thinking skills among students (Currie, 2023; Yeo, 2023). Additionally, the concern that students may not do much research when writing their academic papers has also come up; with ChatGPT, students can generate an entire article without typing a word. Another serious concern for educators is that students with ChatGPT can generate fictitious datasets for their research works without going to the field to collect data. An exciting but worrying thing about using ChatGPT is that ChatGPT's contents can bypass plagiarism detectors due to their generative AI feature. The generative AI feature of these Chatbots allows them to produce unique and original content; at almost all times, this will bypass similarity checks. So, a student can even score a zero-similarity index against plagiarism and not contribute to the writing of the content. Also, studies have shown that AI detection tools can sometimes produce false positives (Elkhatat et al., 2023). In addition, many methods of evading AI detection exist; AI generative tools and their constantly evolving sophistication make detecting content generated by AI or humans more challenging. This phenomenon requires educators and policymakers to develop regulations to guide the use of generative AI tools among students.
Lastly, there are concerns about the erosion of authentic communication. While the chatbot excels at simulating a human-like conversation, its excessive use could contribute to students' over-reliance on pre-generated responses and potentially hinder the development of interpersonal skills and the ability to express original thoughts. The fear is that the excessive use of ChatGPT amongst students could diminish genuine human interaction in educational settings, leaving students with a skewed understanding of effective communication (Baskara, 2023; Rane et al., 2023).
While we acknowledge the ethical issues surrounding the use of ChatGPT amongst students, it is essential always to strike a balance between leveraging its benefits for education and safeguarding against the unintended consequences that it poses. Successfully navigating the ethical complexities associated with using ChatGPT amongst students requires careful consideration, regulation, and a commitment to preserving the core values of education; thus, we must view this tool as a threat and an opportunity for innovation. This calls for dialogue amongst educators and policymakers in shaping policies that ensure the responsible use of these AI tools in the academic realm.
Transforming Examination Writing in Higher Education with ChatGPT
Examination writing is a crucial assessment activity in the education process and allows for evaluating students’ knowledge, skills, and analytical abilities, which are essential for educational growth (Astin, 2012; Brown et al., 2013). One of the ways of transforming examination writing is to incorporate the use of technology in its process. Enhancing such a core educational component with technology aligns with the goals of improving pedagogy through technology, as outlined by Swan et al. (2006) and Jaap et al. (2021), just as online examination writing provides a digital platform for examination writing and assessment (Ardid et al., 2015; Akimov & Malin, 2020). Transforming examination writing thus refers to changing or improving the way examinations are structured; this includes drafting examination questions, writing examinations, and streamlining the assessment process (Hift, 2014; Agius et al., 2017).
The goal is to address the limitations and drawbacks of the traditional format of examination writing, thereby facilitating the examination's effectiveness and ensuring a fair assessment (Wolcott & Legg, 1998; Slavich & Zimbardo, 2012; Hift, 2014; Agius et al., 2017). Thus, transforming examination writing can involve a variety of strategies, such as incorporating different questions of the same level of difficulty of the same content, generating questions that require critical thinking, generating questions with case scenarios, using formative assessment, and finally ensuring inclusiveness which is considering the diverse backgrounds and needs of students when generating examination questions (Lai, 2011; Herrington et al., 2014). This process can be tedious in situations where multiple varied questions must be set with different case scenarios for various groups of people in large numbers, especially if a human should be doing that. This situation calls for adopting technology to facilitate such a process, and an AI chatbot like ChatGPT stands a chance of being up for the task.
In their conceptual paper, Ivanov and Soliman (2023) predict that, by the year 2030, digital teachers like ChatGPT will be able to grade tests, assignments, and other course assessments, giving students prompt and precise results. This phenomenon seems to be already possible; this paper attempts to demonstrate the usefulness of ChatGPT in examination writing. The researchers would like to emphasise that the study does not seek to promote ChatGPT as a replacer for teachers but as a tool that teachers can use to augment their teaching and research capabilities. The researchers acknowledge the irreplaceable role of teachers in education and aim to showcase how AI tools like ChatGPT could be harnessed as a complementary resource to enhance the educational experience for teachers.
Problem statement
Research on ChatGPT and its application in education has emerged since its launch in November 2022. Many of these studies have focused on the ethical implications of ChatGPT in education, examining its usage among students for personalised and adaptive learning and how ChatGPT could be a disruptive tool to the information environment (Biswas, 2023; Cingillioglu, 2023; Elbanna & Armstrong, 2023; Halaweh, 2023; Opara et al., 2023; Qasem et al., 2023; Rahman & Watanobe, 2023). However, little has been done to explore its potential use in transforming examination writing or improving upon students’ assessment; for this reason, this study was carried out.
Statement of the research gap
From the research gap identified, a clear and explicit statement can be given: How can the AI tool ChatGPT help to transform examination writing in Higher Education?
Objectives of the study
The study objectives were carefully designed after thoroughly reviewing the literature on transforming examination writing and consultation with some educationists. Combining the expert views and opinions of practising professionals and published works was necessary to develop well-deserving objectives for the study to answer the research gap statement. Going by that, the study sought to pursue the following objectives:
-
To task ChatGPT to generate varied questions of the same level of difficulty on a subject matter.
-
To task ChatGPT to generate varied questions on a subject matter with case scenarios.
-
To task ChatGPT to generate varied questions on a subject matter that requires critical thinking.
-
To task ChatGPT to generate varied questions on a subject matter that take into consideration the diverse needs and backgrounds of students.
-
To task ChatGPT to help streamline the assessment process.
These objectives, when explored, will help answer the research question by showing the diverse ways ChatGPT can be used to transform examination writing in Higher Education, thereby addressing the gap in the literature.
Justification of the objectives
The paper justifies the above objectives by providing the rationale for its formulation as rooted in scholarly works. Objective number one is supported by evidence from the work of Brown and Race (2012), which shows that varied assessment questions better evaluate student learning outcomes and reduce student anxiety. This is further supported by Pereira et al. (2023), adding that varied questions allow for a more comprehensive evaluation of students’ knowledge. Using case scenarios in examination questions allows for contextualising in the real world; this offers an effective way to assess students in a higher order of thinking as espoused by Casey et al. (2009) and Smith & Mucklow (2016); subsequently, this justifies objective number two. Objective three targets critical thinking skills widely considered necessary in learning, especially in Higher Education (Wals & Jickling, 2002; Behar-Horenstein & Niu, 2011; Liu et al., 2014). The Higher Education process thus requires students to interpret, analyse, evaluate, explain, and infer (Facione, 1990). A chatbot that can assist with this will offer significant relief to educators. Objective four addresses equity and fairness, pivotal aspects driving educational reforms. An assessment process that considers students' diverse backgrounds will ensure that some sections are not disadvantaged in the examination process (Benadusi, 2001; Gilbert et al., 2011; Ling & Nasri, 2019). The assessment process can be tedious, especially when the number of students is significant; therefore, the need for a shared assessment practice is advised (López-Pastor & Sicilia-Camacho, 2017). The assessment process also requires promptly providing high-quality, actionable feedback to students (Pereira et al., 2016). Therefore, having a chatbot to assist with this process will significantly benefit educators.
Contextual scope of the study
The study focus of the examination transformation is narrowed to setting examination questions and streamlining the assessment process. The study is more concerned with the automated generation of examination questions and marking schemes for assessment using ChatGPT.
Significance of the study
The introduction of ChatGPT has opened the way for researchers to engage in the academic discourse regarding the prospects of the tool. Rudolph et al. (2023) highlight the importance of ChatGPT as a tool to aid students in learning and a tool that would lessen teachers' workloads. In tourism education and research, ChatGPT has been found to have the potential to revolutionise this area, as accounted for by Ivanov and Soliman (2023b). Despite the ethical concerns about academic integrity, Adiguzel et al. (2023), in their study, “Exploring the transformative potential of ChatGPT”, re-iterates that the tool can significantly help students and teachers in academia. Benuyenah (2023) backs this claim with the submission that despite all the challenges of AI, “we must acknowledge that Higher Education will thrive alongside any AI evolution as long as we learn to adapt our pedagogy and assessment strategies”. This is in line with the position of this paper, that notwithstanding the challenges and concerns, AI, when used responsibly, can significantly benefit the educational sector. This is why this study was conducted to re-iterate the benefit of AI in education, specifically the use of ChatGPT.