Detecting Outliers from Quiz Response Times and Scores by Classifying Weekly Learning Patterns in a Blended Learning Course
DOI:
https://doi.org/10.52731/liir.v005.260Keywords:
classification, learning analytics, learning log, learning management system, outlierAbstract
Learning management systems are now widely used in many classes, and learning logs are accu-mulated daily, making it increasingly important to mine knowledge and data for learning analyt-ics. In this study, a new experimental class was conducted to collect and analyze Moodle learning logs in a blended learning course in which 57 university students had pre-enrolled.Weekly learn-ing analytics were conducted focusing on quiz answer times and scores. Furthermore, the clus-tering heatmaps were generated to visualize the transition of learning status. Outliers were iden-tified using the interquartile range, 3σ method, and Mahalanobis’ generalized distance. The ex-perimental results observed that there were weeks when outlier learners appeared, whereas in other weeks, they did not.Although the majority of learners did not fall under the outlier category, it became clear that some learners were identified as outliermultiple times. Learners who repeat-edly fall into the outlier category are at risk of encountering challenges in their learning, so early academic intervention is desirable. Clustering heatmaps and outlier visualizations, which depict learning status, are influenced by attendees' learning motivation and prerequisite knowledge, re-sulting in varying outcomes across different classes. By performing outlier detection according to this paper every week, teachers can easily discover learners who are having trouble learning.
References
G. Lampropoulos, “Educational data mining and learning analytics in the 21st Century,” En-cyclopedia of data science and machine learning, IGI Global, 2023, pp.1642-1651.
E. Chung, N.M. Noor, V.N. Mathew, “Are you ready? An assessment of online learning read-iness among university students,” International Journal of Academic Research in Progressive Education and Development, vol. 9, no. 1, 2020, pp. 301-317.
M.C. Sáiz-Manzanares, R. Marticorena-Sánchez, “Monitoring students at the university: De-sign and application of a moodle plugin,” Applied Sciences, vol. 10, no. 10, 2020, 3469.
N. Valle, P. Antonenko, K. Dawson, “Staying on target: A systematic literature review on learner‐facing learning analytics dashboards,” British Journal of Educational Technology, vol. 52, no. 4, 2021, pp. 1724-1748.
O. Bulut, G. Gorgun, H. Karamese, “Incorporating Test‐Taking Engagement into Multistage Adaptive Testing Design for Large‐Scale Assessments,” Journal of Educational Measurement, 2023, pp. 1-24. DOI: 10.1111/jedm.12380
B.C.E. Oguguo, F.A. Nannim, J.J. Agah, “Effect of learning management system on Student’s performance in educational measurement and evaluation,” Education and Information Tech-nologies, vol. 26, 2021, pp. 1471-1483.
J. Kabathova, M. Drlik, “Towards predicting student’s dropout in university courses using different machine learning techniques,” Applied Sciences, vol. 11, no. 7, 2021, 3130.
G. Deeva, J. De Smedt, P. De Koninck, J. D. Weerdt, “Dropout prediction in MOOCs: a comparison between process and sequence mining,” In Business Process Management Work-shops: BPM 2017 International Workshops, Barcelona, Spain, September 10-11, 2017, Re-vised Papers 15, 2018, pp. 243-255. Springer International Publishing.
R. Asad, S. Altaf, S. Ahmad, H. Mahmoud, S. Huda, “Machine learning-based hybrid ensem-ble model achieving precision education for online education amid the lockdown period of COVID-19 pandemic in Pakistan,” Sustainability, vol. 15, no. 6, 2023, 5431.
H. Pallathadka, A. Wenda, E. Ramirez-Asís, “Classification and prediction of student per-formance data using various machine learning algorithms,” Materials today: proceedings, vol. 80, 2023, pp.3782-3785.
M.D. Ariastuti, A.Y. Wahyudin, “Exploring academic performance and learning style of un-dergraduate students in English Education program,” Journal of English Language Teaching and Learning, vol. 3, no. 1, 2022, pp. 67-73.
S.L. Wise, “An information-based approach to identifying rapid-guessing thresholds,” Ap-plied Measurement in Education, vol. 32, no. 4, 2019, pp. 325-336.
S.L. Wise, M.R. Kuhfeld, “A cessation of measurement: Identifying test taker disengage-ment using response time,” Integrating timing considerations to improve testing practices. Routledge, 2020. pp.150-164.
S.L. Wise, “Rapid‐guessing behavior: Its identification, interpretation, and implications,” Educational Measurement: Issues and Practice, vol. 36, no.4, 2017, pp. 52-61.
S.L. Wise, X. Kong. “Response time effort: A new measure of examinee motivation in com-puter-based tests,” Applied Measurement in Education, vol. 18, no. 2, 2005, pp. 63–183.
H. Guo, J.A. Rios, G. Ling, Z. Wang, L. Gu, “Influence of Selected‐Response Format Vari-ants on Test Characteristics and Test‐Taking Effort: An Empirical Study,” ETS Research Re-port Series, vol. 2022, no. 1, 2022, pp. 1-20.
P. Mishra, C.M. Pandey, U. Singh, A. Gupta, “Descriptive statistics and normality tests for statistical data." Annals of cardiac anaesthesia, vol. 22, no. 1, 2019, pp. 67-72.
P. Govender, V. Sivakumar, “Application of k-means and hierarchical clustering techniques for analysis of air pollution: A review (1980–2019).” Atmospheric pollution research, vol. 11, no. 1, 2020, pp. 40-56.
C. Boutsidis, P. Drineas, “Unsupervised feature selection for the k-means clustering prob-lem,” Advances in neural information processing systems, vol. 22, 2009.
A Boukerche, L Zheng, O Alfandi, “Outlier detection: Methods, models, and classification,” ACM Computing Surveys (CSUR), vol. 53, no. 3, 2020, pp.1-37.
K. Dobashi, C.P. Ho, C.P. Fulford, M.F.G. Lin, “A heat map generation to visualize engage-ment in classes using moodle learning logs,” In 2019 4th international conference on infor-mation technology (InCIT), IEEE, 2019, October, pp. 138-143.
O. Bulut, G. Gorgun, T. Wongvorachan, B. Tan, “Rapid guessing in low-stakes assessments: Finding the optimal response time threshold with random search and genetic algorithm,” Al-gorithms, vol. 16, no.2, 2023, 89.
