Machine Learning-Based Real-Time Feedback Assessment System for Student Performance Prediction in Tertiary Institution

doi:10.21203/rs.3.rs-4923469/v1

Download PDF

Research Article

Machine Learning-Based Real-Time Feedback Assessment System for Student Performance Prediction in Tertiary Institution

https://doi.org/10.21203/rs.3.rs-4923469/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

The need for effective and digitized formative feedback mechanisms in classroom management of core courses in tertiary institutions in the developing world such as Nigeria, Kenya, and Ghana is paramount. A fair trivial environment is needed for students to learn and interact with their tutor effectively. This paper presents a framework for feedback assessment in real-time for student performance prediction using a machine learning approach in the university to maximize students’ satisfaction through an internalized and effective learning environment by monitoring students’ level of engagement during lecture sessions. The analysis from the existing system shows that the large amount of data generated from students’ responses makes it possible to predict student performance per course. This was done using machine learning (K-Nearest Neighbor) to predict the likelihood of student performance and engagement overtime on the dataset generated from attendance, personal and assessment history. The system was developed using Django (Python Framework). The empirical result from the classifier shows that KNN presented an accuracy of 78%. The implication of the study would further assist the developing country’s university system, increase the performance rate of student engagement and lecturer’s teaching styles, as well as aid in the educational decision-making process.

Feedback

Machine Learning

Performance Prediction

Real-time

Tertiary Institution

Information and communication technologies (ICTs) have redefined our world, it has led to the advancement in industries, business transactions, educational and health sectors such that quality of life is being assessed by how these sectors are fully integrated with this growing innovation [1]. This has resulted in strong consequences in the learning sector around the world, with the increased use of laptops, smartphones and tablets by staff and students for learning [2–3]. Also, with the availability of internet connectivity and Wi-Fi networks in some tertiary institutions, staff and students can easily access online resources. It has provided fertile ground for utilizing the deployment of a real-time assessment and prediction feedback system in institutions.

Nonetheless, over the years, lectures have mostly been delivered in developing countries’ university systems such as Nigeria, Kenya, Ghana, etc. in a conventional approach wherein the lecturer presents materials in the lecture format and students submissively gather the materials by listening and taking notes. Intermittently, the lecturer may either call on several students to answer questions or use volunteer techniques to ascertain their level of understanding. This method of getting feedback does not consider the shy students as the sample size or volunteers chosen are normally dominated by outstanding and outspoken students. Also, considering the large number of students offering core courses in institutions, it is difficult for the lecturers to accurately assess and evaluate the general level of engagement or understanding of what is being taught, and as a result, many end up not giving immediate feedback and reviews. Other means of measuring students’ understanding of concepts which include quizzes, assignments and examinations do not still present timely feedback, hence, the need for this system [1–4]. Feedback being a powerful factor of learning according to [5] must be immediate or timely before it can be considered effective in stimulating better knowledge. Thus, Real-Time feedback systems can assess teaching and learning methods instantaneously, especially in classes with large enrollments of students [5–7]. In addition, proper evaluation of a lecturer’s teaching style has been a major challenge for institutions due to the bias of the responses from students; either because of hatred, victimization or lack of appropriate information about the lecturers. This automated system of getting feedback will of course encourage anonymous feedback from students, ascertain the number of students engaging in a course, track students’ performances and allocation of courses to lecturers based on the performance of students; this will in turn improve effective learning and teaching style cum philosophy.

Furthermore, the vast amount of educational data arising from assessments encompasses hidden information which makes prediction difficult. Prediction of students’ performance has a way of accomplishing a higher level of quality in higher education [7–9]. To extract this relevant information, a data mining technique is used. Data mining is a machine-learning technique that involves extracting knowledge as well as patterns from large amounts of data [10–13],[27]. This played a significant role in improving student learning. For instance, a recent study by [28] developed a regression model using machine learning methods for educational data mining to evaluate and forecast secondary school student performance utilizing 30 attributes from academic history data. For increased accuracy, though, more prediction models are needed. Also, [29] employed data mining based on artificial intelligence (AI) to predict student performance and learning analytics in online engineering courses. The study enhances the quality of the student's learning and provides timely and ongoing feedback.

Consequently, the objective of this study is to develop an automated formative feedback system that will be able to assess, evaluate and predict students’ performance using machine learning techniques. The study bridges the gap between students' and lecturers’ relationship by developing a real-time feedback application meant to address the inherent issues relating to students' understanding during lecture sessions, active participation and engagement in class, performance prediction and evaluation of lecturers’ teaching style. This is done using a data mining algorithm that has the capability of predicting future occurrences such as the KNN algorithm. The prediction is based on the information stored on individual students such as assessment scores and students’ personal information, as datasets. Specifically, our knowledge that contributed to the novelty of this study includes:

Provides an in-depth discussion on a real-time feedback system using machine learning algorithms to analyze student performance during lecture sessions;
We designed an automated formative feedback system that allows assessment, evaluation and prediction of students’ performance using machine learning techniques;
We Implemented the designed framework that assesses, evaluates and predicts students’ performance using machine learning techniques;
Also, we provide a web platform to evaluate and revalidate students' and lecturers’ relationships in the classroom using a real-time feedback system;
We further predicted and evaluate the performance of students during lecture sessions, active participation and engagement in class, and lecturers’ teaching style.

The remaining part of this work is organized into different sections as follows; section 2 reviews related literature, the architecture of the proposed system is discussed in section 3, and section 4 presents the results and discussion. Finally, the conclusion and future research were highlighted in section 5.

Real-time feedback systems can be found in many kinds of literature using different names including online class response systems [1], i-clickers [2, 9], audience response systems [3], anonymous formative feedback systems [8] and smart-based student response systems [23], Online Student Response System [10]. Most of these systems are very similar in function; they require questions from lecturers through any transmitting device, a response from students through a receiving device; and software that collates the information (responses), in a convenient and interpretative form [4–6]. While some authors surveyed different target audiences using different parameters to ascertain the usefulness of the system on students’ engagement and performance in class, others designed a similar system to handle the issues of survey systems with its benefit to students [7–8]. The result of the research carried out was found to be useful to students’ engagement and performance in class [1, 4, 9–10, 23] but recorded some gaps including; obscene submissions from students [23], poor implementation [1], non-testability [7], privacy issues [8], student-centric with no timely feedback [9], inefficiency of online survey systems and inability to monitor students’ performance together with prediction feature.

In recent years, a significant amount of literature has focused on predicting student performance using machine learning techniques [12, 18, 21, 25]. While some authors reviewed different machine learning algorithms [11], others applied different classifiers/data mining algorithms in the prediction of performances (pass or fail and grade) using different parameters. Some of the classifiers considered were decision trees, Naive Bayes Neural networks [14–16], KNN [18–19], Naive Bayes, Multi-layer perception (MLP), J48, and REPTree [22], Linear regression, Neural Network and Support vector machine (SVM) [22, 24, 26], etc. This research were carried out on datasets generated from different sources ranging from real-life datasets, online machine learning libraries etc. The results from the research show that one algorithm may have a better accuracy/efficiency considering the nature of the classifier, size of datasets, errors introduced, training set and time. Amongst the classifiers applied, Linear Regression [22], SVM [22, 24], MLP [17], and Naive Bayes [15] came out best while hybrid approaches (classifiers) give better accuracy [13], also [25] found out that students with better performance in Math’s courses have a better chance of excelling in Computer Science.

Additionally, machine learning algorithms [30], have been widely utilised in higher institutions to improve the academic performance prediction of students and learning analytics to a quick feedback mechanism. For instance, [28], used regression models of machine learning techniques in conjunction with educational data mining to evaluate and forecast secondary school student performance utilizing 30 attributes from academic historical data. But for greater accuracy, more predictive models are needed. Also, [31] demonstrates data mining based on artificial intelligence (AI) to forecast student performance and learning analytics in online engineering courses. The study enhances students' learning quality and provides timely, ongoing feedback. However, additional sample data are needed to support the author's assertion. This project seeks to solve the challenges identified above by developing an automated formative feedback system that will be able to assess, monitor and as well incorporate prediction of student’s performances/engagement over time using the KNN algorithm. The algorithm was chosen because of its features of understandability with better prediction accuracy when compared to other algorithms considering the datasets involved and the classifiers reviewed.

This part of the system was designed using Django (Python framework) on a Python IDE known as PyCharm because it has the capability of communicating with Python programming for the prediction aspect, web development and connecting to databases. We have students, staff and admin as the major actors. Figure 1 below depicts the role an actor can play in the input data box. All information from the input box is manipulated and processed at the middle tier, stored in the data tier (database) and then sent an output (output data tier) for the user’s view. Every user of the system must be registered by an admin and subsequently log in with their username and password. It is expected that at the end of a lecture session, the lecturer should post timed questions for students to answer, this is to enable him to ascertain how well his lecture was understood. The questions are timed to curtail online cheating. The students eligible to take any test must have registered for the course with the lecturer via this app. The test scores generated were calculated by summing the total number of correct answers divided by the number of questions given and then multiplied by 100. The general class score is presented on a bar chart for easy interpretation. The chart groups student’s scores accordingly using the 5-point grading scale. Assessments of students were carried out by calculating the average test score by summing every test score of students and dividing by the number of tests taken per course. Also, the evaluation of staff was carried out by students answering all the 20 questions presented, where each question carries 5 marks, summing up to 100 (20x5) marks. The idea behind including students’ assessment scores in the prediction was to discourage offensive submissions and as well inactive participation in the assessment exercise.

3.1 Description of dataset

This research applied KNN in the prediction of student’s performance over time in a course. A total of 395 rows x 31 columns dataset were used for training the system in predicting students’ performance of which 62% of the data were used as a training set and 38% as a test set. The data set used is from the UCI Machine Learning Repository; (https://archive.ics.uci.edu/ml/datasets/Student+Performance). The parameter used for the prediction of student performance is based on the output from the assessment system and other inherent personal information (100%). Some of the inherent information was in string format but had to be converted to numerical values. These datasets which were collected both in string and numeric form were prepared through rescaling and normalization to match the needed standard for our prediction. The assessment scores together with the students’ inherent information stored in the database are called upon whenever the prediction button is activated. This information includes: school, sex, age, family size, family support, health status, study time, travel time, passed, internet free time and so on.

3.2 Prediction Implementation

The prediction aspect of the work was implemented using a weighted KNN approach algorithm. For each missing grade, the ten most similar students were considered in terms of the Pearson correlation coefficient and a weighted average of other closely related students’ scores to fill in the missing grade value. Due to some data being qualitative and relative to the students’ perceptions, these limitations of our dataset were kept as potential causes of error. Python programming was used for the implementation wherein the Numpy and Pandas libraries were used for data manipulation. Matplotlib and Seaborn libraries in Python were used along with ggplot2 to create data visualizations. This Python programming written on a Django framework (PyCharm) was implemented on an XAMPP server. To determine a baseline for acceptable predictor performance, the naïve method of using the mean/median value from the training set for each attribute to predict the test set was initiated. The model tested includes KNN from the Scikit-learn library.

3.2.1 KNN implementation steps

The value of k was chosen to be 7; Euclidean distance was selected as our distance metrics; we initialized our weights in case of a tie; feature scaling and normalization were performed, and thereafter trained our data and got a trained model; this model was then used to classify the students in the test data set. Finally, the web application which was created with Python/Django framework accepts inputs relating to the students and with a combination of the trained model can predict students’ likelihood of performance over time. The model classified the expected result from 0–39; 1 = Fail, 40–49; 2 = Pass, 50–59; 3 = Good, 60–69; 4 = Very good and 70–100; 5 = Excellent.

Table 1 and Fig. 2 depict the graphs generated from the model used to evaluate the classifier's performance in training the system. Figure 2 shows the student's learning curve which was used in training the system to avoid underfit or overfit.

Table 1

Students’ prediction results analysis
Model	Accuracy	F1 -Score	ROC_AUC Score	Random_ State	Time
KNN	78%	0.71	0.8	71027464	0.059962511….

From the result presented in Table 1, the outcome of the prediction result above shows that KNN presented an accuracy of 78%; f1_score of 71%; Roc_auc_score of 80% and Random_state of 71027464.

The screenshots in Fig. 3 are some of the results from our feedback application which was tested in modules. We captured the timed assessment questions, general class test results chart representation and the prediction of students’ performance. Others that were not captured here include, the login page, register page, course allocation page, department and course addition page, questions addition and time setting page etc. The captured pages are shown in Fig. 3. It shows the course allocation page of the system. It captures the names of staff, and the courses allocated to them by the admin whose role includes allocating courses to staff.

Figure 4 shows the screenshot of the assessment questions asked at the end of a lecture session. It is expected that every student will log into his/her page at the set time to answer the posed questions. The questions are close ended. As soon as the set time elapses, the test is terminated.

Figure 5 above shows the screenshot of the general performance of a class in a chart form. Each performance is represented in a rectangular bar which is differentiated using distinct colors as seen.

Also, Fig. 6 shows the screenshot showing the likelihood of a student passing or failing a course at the end of a course. It has the actual and predicted grade column together with other information identifying a student. The actual is the current performance while the predicted is the estimated performance.

Feedback is a very crucial aspect of learning, as students can easily monitor their learning outcomes. Feedback when provided in real-time gives more clarifications to students on what they understand best or fail to understand thus giving room for follow-up and improvement. It is indicated from the reviews conducted that many assessments in classroom settings are carried out in a traditional form of which there are flaws accompanied with it. However, this system is designed to address the gap identified by these achievements. The implemented system has been able to facilitate real-time assessment of students online. This will go a long way toward encouraging students’ participation in class and as well increase retention, the analysis of students’ performance in a chart form, will enable easy interpretation briefly as a result saves useful time for the staff and students as well. The prediction of performances will help to improve the performance of students as they stand to know their possibility of passing or failing a course before examination. Generally, this system improves students' learning outcomes and encourages self-assessment to create room for better academic performance. The prediction result from the KNN classifier presented an accuracy of 78%. The implication of the study would be utilized by developing the country’s university system, increasing the performance rate of student engagement and lecturer’s teaching styles as well as aiding in the educational decision-making process.

5.1 Suggested areas for future work

However, this research was not without difficulties. Such limitations as to the accessibility of real-life data for analysis and time constraints were encountered. Thus, the following recommendations are put forward for future work; the system should be used in various departments to ascertain the level of its usefulness, flaws, cost-effectiveness and availability of data. Finally, different machine learning algorithms should be used in the prediction of performances to ascertain the best algorithm that will yield a better result.

Ethical Statement: To our knowledge, the submitted work should be original and has not been published elsewhere in any form or language.

Competing Interest: There is no competing interest of any sort

Funding: There is no external funding received by the authors, though, sought for full waiver of the article processing charges from the Springer Nature

Conflicts of Interest: The authors declare no conflict of interest both financial and non-financial for this article.

Availability of data and material (data transparency): Available on request

Code availability (software application or custom code): Available on request

Consent to participate: Not Applicable

Consent for publication: Not Applicable

Contributions: DUE developed the structure of the paper, worked on the critical revision of the article, and gave final approval of the article. In addition, ACI handled the data analysis and interpretation and participated in the grammatical check and critical revision of the article.

CJE prepared the article and participated in the critical revision of the article as well as contributed to the research concept and design and was involved in data collection. More so, CVI participated in graphical sketching, proofreading, and Grammarly checks.

L. Sun, “The use of a real-time online class response system to enhance classroom learning," ASEE Engineering Design Graphics Division - 69^th Midyear Conference, Normal, IL, 2014, http://commons.erau.edu/publication/170
K, Egelandsal and R. J Krumsvik, “Clickers and formative feedback at university lectures” Education and Information Technology Journal, 22:55-74, 2017, Doi: 10.1007/s10639-015-9437-x
S. Hepplestone, I. Glover, B. Irwin, and H. Parkin, “Setting out the role of feedback in the assessment process through both the student and tutor perspective” Practitioner Research in Higher Education vol. 10, no. 1, pg. 81–90, 2016, Available: http://shura.shu.ac.uk/16585/.
M. A. Sarvary, and K. M. Gifford, “The benefits of a real-time web-based response system for enhancing engaged learning in classrooms and public science events” The Journal of Undergraduate Neuroscience Education, vol. 15, no. 2, 2017, E13-E16.
T. B. Keinding, and A. Qvortrup “Feedback as Real-time Constructions” E-Learning and Digital Media, vol. 11, no. 2, 2014, http://dx.doi.org/10.2304/elea.2014.11.2.191
B. Ada, “Using design-based research to develop a mobile learning framework for assessment feedback” Research and Practice in Technology Enhanced Learning, vol. 13, no. 3. 2018, [Online], https://doi.org/10.1186/s41039-018-0070-3
A. W. Mogwe, “Real–time concept feedback in lectures for Botho University students”, International Journal of Advanced Computer Science and Applications, Vol. 9, no. 6, 2018, pg 246-251.
A. Avic, P. Gronberg, J. Olsson, and G. Riveros, “Student and Teacher Response System: Development of an interactive anonymous real-time formative feedback system”, 2013.
T. Stelzer, M. Selen and G. Gladding, “clickers”, Macmillan Learning, 2005, Available at: https://www.iclickers.com
M..H. Abdulla “The use of an online SRS to support lectures to medical students” Education and Information Technology Journal, 2018, Doi: 10.1007/s10639-018-9752-0
D. Sharma, and N. Kumar, “A review on machine learning algorithms, tasks and applications”, International Journal of Advanced Research in Computer Engineering & Technology (IJARCET), Vol. 6, no. 10, October 2017, ISSN: 2278 – 1323.
W. Li, D. Mu˜noz, A. Richert, and C.S. Tucker, “Data Mining Driven Approach To Real-Time Classroom Feedback Through Multimodal Sensing, The Pennsylvania State University”, 2017 Available on: http://nsse.iub.edu
A. Kaur, and N.U. Sing, “Machine learning approach to predict academic performance” International Journal for Research in Applied Science and Engineering Technology (IJARSET), Vol. 6, no. 4, 2018, Available at https://www.ijraset.com
A. M Shahiria, W. Husaina, and N.A. Rashida, “A Review on Predicting Student’s Performance using Data Mining Techniques”, “Procedia Computer Science” vol.72, no. 414 – 422, 2015, doi: 10.1016/j.procs.2015.12.157, Available online at www.sciencedirect.com
E. Osmanbegović, and M. Suljić, “Data mining approach for predicting student performance” Journal of Economics and Business, vol. 10 no. 1, 2012.
R. A. Usiobaifo, and R.O. Osaseri, “A machine learning approach for predicting postgraduate students’ performance” proceedings of INCEDI 2016 Conference Accra, Ghana, pg 779-784, ISBN: 978-9988-2-3994-7, 2016.
P. Kaur, M. Singh, and G. S. Josan, “Classification and prediction based data mining algorithms to predict slow learners in the education sector” 3rd International Conference on Recent Trends in Computing, vol. 57(500-508), 2015.
I. S Alfere, and A.Y. Maghari, “Prediction of student's performance using modified knn classifiers”1st International Conference on Engineering & Future Technology (ICEFT 2018)At University of Palestine, Gaza, Palestine, pg: 143
S. B. Imandoust, and M. Bolandraftar, “Application of k-nearest neighbor (knn) approach for predicting economic events: theoretical background”, Journal of Engineering Research and Applications, Vol. 3, Issue 5, pp.605-610Available at: www.ijera.com
A. Kataria, and M. D. Singh, “A review of data classification using k-nearest neighbour algorithm” International Journal of Emerging Technology and Advanced Engineering, ISSN 2250-2459, Vol. 3, no. 6, 2013, Available: www.ijetae.com
S. Aydogdu, “Predicting students’ final performance using artificial neural networks in online learning environment”, Education and Information Technology Journal, 2019, Doi: org/10.1007/s10639-019-10053-x.
E. Y. Obsie, and S. A. Adem, Prediction of student academic performance using neural network, linear regression and support vector regression: A case study, International Journal of Computer Applications (0975 – 8887), vol.180, no.40, 2018.
S. McLoone, and C. Brennan, “A smart-phone based SRS for obtaining high-quality real-time feedback – evaluated in engineering mathematics classroom” School of Electronic Engineering, Dublin City University, 2013, Available at www.dcu.ie
K. Lee, K. Kim, J. Kan, S. Choi, Y. Im, Y. Lee, and Y. Lim, “Comparison and analysis of linear regression & artificial neural network” International Journal of Applied Engineering Research, ISSN 0973-4562 Vol. 12, no. 20, 2017, pp. 9820-9825, Available at: http://www.ripublication.com
O. D. Oyerinda, and P. A. Chia, “Predicting students’ academic performances – a learning analytics approach using multiple linear regression” International Journal of Computer Applications (0975 – 8887) Vol. 157, No. 4, 2017.
S. Al-Sudani and R. Palaniappan, “Predicting student’s final degree classification using extended profile”, Education and Information Technology Journal, 24:2357-2369-74, 2019, Https://doi.org:/10.1007/s10639-019-09873-8
A. C. Ikegwu, H. F. Nweke, C. V. Anikwe, U. R. Alo, and O. R. Okonkwo, “Big Data Analytics for Data-driven Industry: A Review of Data Sources, Tools, Challenges, Solutions and Research Directions,” Clust. Comput., vol. 25, no. 5, pp. 3343–3387, 2022, doi: https://doi.org/10.1007/s10586-022-03568-5.
S. Hussain and M. Q. Khan, “Student ‑ Performulator : Predicting Students ’ Academic Performance at Secondary and Intermediate Level Using,” Ann. Data Sci., vol. 10, no. 3, pp. 637–655, 2023, doi: 10.1007/s40745-021-00341-0.
F. Ouyang, M. Wu, L. Zheng, L. Zhang, and P. Jiao, “Integration of artificial intelligence performance prediction and learning analytics to improve student learning in online engineering course,” Int. J. Educ. Technol. High. Educ., vol. 20, no. 1, pp. 1–23, 2023, doi: 10.1186/s41239-022-00372-4.
A. C. Ikegwu, F. H. Nweke, and C. V. Anikwe, “Recent trends in computational intelligence for educational big data analysis,” Iran J. Comput. Sci., vol. 7, no. 1, pp. 103–129, 2024, doi: 10.1007/s42044-023-00158-5.
F. Ouyang, M. Wu, L. Zheng, L. Zhang, and P. Jiao, “Integration of artificial intelligence performance prediction and learning analytics to improve student learning in online engineering course,” Int. J. Educ. Technol. High. Educ., vol. 20, no. 1, pp. 1–23, 2023, doi: 10.1186/s41239-022-00372-4.

No competing interests reported.

Download PDF

Editorial decision: Revision requested
05 Nov, 2024
Reviews received at journal
04 Nov, 2024
Reviewers agreed at journal
21 Oct, 2024
Reviewers invited by journal
21 Oct, 2024
Editor assigned by journal
18 Aug, 2024
Submission checks completed at journal
18 Aug, 2024
First submitted to journal
16 Aug, 2024

You are reading this latest preprint version

Machine Learning-Based Real-Time Feedback Assessment System for Student Performance Prediction in Tertiary Institution

Status:

Version 1

Abstract

Figures

1. INTRODUCTION

2. REVIEW OF RELATED LITERATURE

3. METHODOLOGY

3.1 Description of dataset

3.2 Prediction Implementation

3.2.1 KNN implementation steps

4. RESULTS AND DISCUSSION

5. Conclusion

5.1 Suggested areas for future work

Declarations

References

Additional Declarations

Status:

Version 1