Experiments of Automatic Scoring Using Generative AI in a Summary Essay Learning System
DOI:
https://doi.org/10.52731/lir.v004.294Keywords:
e-learning, Japanese essay summarization, natural language processing, automatic scoring, generative AIAbstract
We are developing an e-learning system aimed at improving Japanese language proficiency. This system focuses on the task of summarizing texts. An e-learning system designed for on-demand learning requires an automatic scoring function to provide feedback to users. This paper focuses on the scoring function using generative AI. Generative AI can be utilized in two ways: generating various patterns of correct answers and using it in the scoring process itself. When generating model answers, the user’s responses are compared to these model answers for scoring, so the method of comparison is also considered. This paper reports experimental results on the differences between these applications.
References
T. Yamasaki and A. Hiramatsu, “A Study of Correcting Handwritten Answers for Short Essay Self-learning Systems,” Proc. 14th International Conf. on Learning Technologies and Learning Environments (LTLE 2023), 2023.
T. Ishioka, “Latest Trends in Automated Essay Scoring and Evaluatio,” Japanese Society for Artificial Intelligence, Vol.23, No.1, 2008, pp.17–24.
J. Burstein and M. Wolska, “Toward evaluation of writing style: Finding overly repetitive word use in.student essays, ” Proc. 11th Conference of the European Chapter of the Association for Computational Linguistics (EACL ’03), 2013, pp. 35–42.
E. B. Page, “New computer grading of student prose, using modern concepts and software, ” Experimental Education, Vol.62, No.2, 1994, pp.127–142.
T. K. Landauer, D. Laham, and P. Foltz, “Automated scoring and annotation of essays with the intelligent essay assessor,” Automated Essay Scoring: A Crossdisciplinary Perspective, 2003, pp.87–112.
S. Elliot, “IntelliMetric: From Here to Validity,” Automated Essay Scoring: A Crossdisciplinary Perspective, 2003. pp.71–86.
V. S. Kumar and D. Boulanger, “Automated Essay Scoring and the Deep Learning Black Box: How Are Rubric Scores Determined ?,” Artificial Intelligence in Education, Vol.31, 2021, pp.538–584.
D. Ramesh and S. K. Sanampudi, “An automated essay scoring systems: a systematic literature review,” Artificial Intelligence Review, Vol.55, 2022, pp.2495–2527.
