講演情報
[5O1-IS-1-02]Evaluation of Open Source Large Language Models in Japanese Food-Animal Veterinary Medicine
〇Haruno Fusa1, Chonho Lee1, Yasunori Shinozuka1, Sakuei Onishi1, Hiromitsu Shiina1 (1. OKAYAMA UNIVERSITY OF SCIENCE)
regular
キーワード：
Large Language Models、Food-Animal Veterinary Medicine、Clinical Simulation、Benchmark
Large language models (LLMs) are beginning to be applied in veterinary medicine to support medical data processing, clinical decision support, and disease diagnosis. Among them, open-source models play an important role due to their flexibility for domain-specific development and their advantages in protecting sensitive personal information. However, the availability and performance of open-source LLMs for Japanese food-animal practice remain limited. To address this gap, we establish Food-Animal Clinical QA, a benchmark for evaluating LLMs’ veterinary clinical knowledge in Japanese food-animal medicine. Our evaluation shows that even the recent mainstream open-source LLMs achieve accuracies below 80%, indicating that their performance is still insufficient for practical use in real-world clinical settings. Furthermore, based on our experimental findings, we analyze key directions for future model development. Overall, this study proposes a benchmark for LLMs in Japanese food-animal clinical medicine and contributes to the development of open-source LLMs for Japanese food-animal practice.