[4K5-GS-6c] Language media processing

3:30 PM - 3:45 PM JST(6:30 AM - 6:45 AM UTC)

[4K5-GS-6c-01] Automated Essay Scoring via Learning Pairwise Comparisons

〇Ayako Yamagiwa¹, John Maurice Gayed¹, Masayuki Goto¹ (1. Waseda University)

Keywords:

Pairwise Comparison,Automated Essay Scoring,Embeddings,Large Language Models,Few-shot LearningAutomated Essay Scoring via Learning Pairwise Comparisons

Scoring student essays requires substantial human effort, and automated essay scoring (AES) has therefore been widely studied. Conventional AES approaches mainly train machine learning models on corpora that was scored by teachers or professional evaluators. However, essay evaluation may contain subjective elements, involve multiple evaluation criteria, and often suffers from inter-rater variability and noisy labels. Recently, zero-shot AES using large language models with a method called LLM-based Comparative Essay Scoring has been proposed. Instead of outputting an absolute score, this system ranks then estimates essay scores based on the chosen score mapping from pairwise comparison results between essays, demonstrating the effectiveness of relative evaluation. In this study, we extend this line of research by proposing a framework that trains a model to predict pairwise comparisons of essays and subsequently estimates the evaluation value of each essay. Our approach learns pairwise relationships from data, enabling more stable and systematic score estimation. By focusing on pairwise comparisons rather than direct absolute score prediction, the proposed method can reduce the influence of evaluation noise, even under multiple evaluation criteria. We evaluate the effectiveness of the proposed approach through experiments using the publicly available ASAP 2.0 dataset.