Presentation Information
[4Yin-B-15]Typology of Perceived “Oddness” in LLM-Generated Stories for Elementary School Kanji Learning
〇Kento Yoshimura1, Yuya Sakaguchi2, Kyosuke Takami1 (1. Osaka Kyoiku University, 2. Learn More CO.,LTD.)
Keywords:
Large Language Model,Educational Applications,Japanese Text Quality Evaluation,Kanji Learning
Generative AI (LLMs) is increasingly used in education, yet it remains unclear whether AI-generated Japanese texts meet the quality standards required for instructional use. This study categorizes the types of “oddness” perceived by native Japanese evaluators from an educational perspective. We examined AI-generated instructional materials in the form of short stories, each required to include a specified kanji character. Following national curricular guidelines, we targeted 160 kanji taught in the second grade of elementary school. Under identical prompts, ChatGPT-4o-mini, ChatGPT-4o, and ChatGPT-5 generated 480 stories, which were manually reviewed. Perceived oddness was classified into eight categories: lack of causal relationships, abrupt narrative transitions, inconsistencies in characters or settings, inappropriate kanji usage, improper use of particles or conjunctions, omission of subjects or predicates, unnatural expressions, and excessive use of hiragana. This categorization supports the development of evaluation benchmarks for kanji-learning materials and informs the design of education-oriented LLMs.