[3F1-OS-8] OS-8

10:15 AM - 10:30 AM JST(1:15 AM - 1:30 AM UTC)

[3F1-OS-8-06] Performance Evaluation of Large Language Model for Extracting Treatment, Adverse Events, and Temporal Information from Female Patients with Cancer Social Media

〇Yoshinori Kawaguchi¹, Yuki Nanagisawa¹, Masami Tsuchiya¹, Soma Hisamura¹, Hayato Kizaki¹, Shungo Imai¹, Tomohiro Nishiyama², Eiji Aramaki², Nobuko Ueda³, Tomoko Yodogawa³, Satoko Hori¹ (1. Keio University, 2. Nara Institute of Science and Technology, 3. Peer Ring, General Incorporated Association)

Keywords:

Large Language Model,Social Networking Service,Female Patients with Cancer

Research on applying large language models (LLMs) to extract treatment information and adverse events from social media posts remains limited. This study aimed to evaluate the utility of LLMs for information extraction tasks that identify treatments, adverse events, and their temporal attributes from unstructured and context-dependent posts in the female cancer patient social networking community Peer Ring. We analyzed 100 posts authored by 99 unique patients that described at least one treatment regimen used in neoadjuvant chemotherapy for breast cancer. Using GPT-4.1, we automatically extracted treatment- and adverse event–related event labels, event content, dates of occurrence, and event statuses (e.g., onset and resolution) based on the textual content of the posts. The extraction results generated by the LLM were compared with human-annotated reference data, and performance was evaluated using precision, recall, and F1-score. As a result, F1-scores exceeding 0.9 were achieved across all evaluated items, including those related to temporal information. This study provides a foundational methodology for visualizing the temporal trajectories of treatment courses and adverse events derived from patient-generated social media posts and demonstrates the usefulness of LLMs for information extraction tasks in this context.