講演情報
[2Yin-B-21]Exploring Language Understanding Challenges in Online Texts
〇Zheng Lin Chia1, Krishna Prasad Bhattarai1 (1. Kanazawa Gakuin University)
キーワード：
AI、machine translation、figurative language
Online texts such as social media posts and short-form user-generated content present persistent challenges for machine learning–based language understanding systems.
Although large language models (LLMs) achieve strong performance on standardized benchmarks, it remains unclear whether such improvements reflect robust handling of context-limited and pragmatically rich online discourse.
This work proposes three structural stress conditions in online language: informal domain shift, context underspecification, and pragmatic inversion.
We argue that these conditions systematically expose evaluation blind spots in benchmark-centered LLM assessment.
We argue that linguistically grounded stress-test benchmarks are necessary to more accurately assess real-world robustness, particularly when these conditions compound (e.g., pragmatic inversion under underspecification).
Although large language models (LLMs) achieve strong performance on standardized benchmarks, it remains unclear whether such improvements reflect robust handling of context-limited and pragmatically rich online discourse.
This work proposes three structural stress conditions in online language: informal domain shift, context underspecification, and pragmatic inversion.
We argue that these conditions systematically expose evaluation blind spots in benchmark-centered LLM assessment.
We argue that linguistically grounded stress-test benchmarks are necessary to more accurately assess real-world robustness, particularly when these conditions compound (e.g., pragmatic inversion under underspecification).