[2Yin-B] Poster 4

[2Yin-B-09] Verification of Annotation Capabilities for Harmful Texts Using Large Language Models

〇Rin Miura¹, Michal Ptaszynski², Fumito Masui², René Meléndez³, Yuta Nakajima³ (1. Faculty of Engineering, Kitami Institute of Technology, 2. Division of Information and Communication Engineering, Kitami Institute of Technology, 3. Graduate School of Engineering, Kitami Institute of Technology)

Keywords:

Large Language Models,Detection of harmful information,Japanese Texts Classification

The proliferation of social networking services (SNS) has led to problems arising from the spread of harmful information. Japanese datasets for harmful information, essential for developing automated detection technologies required for healthy platforms, are scarce and costly to construct. This study compared and evaluated the annotation capabilities of large language models (LLMs) for SNS text to reduce costs. We analyzed trends using information content based on vocabulary density and text volume. While cloud LLMs demonstrated high performance, local LLMs also showed improved performance when using few-shot prompts. Furthermore, performance degradation was observed as information content decreased, highlighting the challenge of improving performance at medium information content levels and below.