[4K5-GS-6c] Language media processing

3:45 PM - 4:00 PM JST(6:45 AM - 7:00 AM UTC)

[4K5-GS-6c-02] Translation Quality Evaluation Using Large Language Models Incorporating Legal Translation Guidelines

〇Kota Kawano¹, Isao Goto¹, Takashi Ninomiya¹ (1. Ehime University)

Keywords:

Large Language Models,Legal Translation,Translation Quality Evaluation,Error Detection

Legal translation requires a high level of accuracy, as it deals with specialized documents. While practical translation guidelines are essential in this domain, there has been limited research on systematically evaluating whether assessment methods can detect guideline violations. This study proposes a meta-evaluation framework for legal translation assessment. We construct a benchmark dataset consisting of translation examples that violate individual items of the Guidelines for the Translation of Japanese Laws into English, enabling item-wise measurement of violation detection performance. As a case study, we implement an LLM-based error detection method and evaluate it within the proposed framework. Experimental results show that explicitly providing the guidelines improves detection rates, demonstrating that the framework can quantitatively compare evaluation methods based on their ability to identify guideline violations.