[2H5-OS-2b] OS-2

3:45 PM - 4:00 PM JST(6:45 AM - 7:00 AM UTC)

[2H5-OS-2b-02] Characteristic Analysis and Out-of-sample Test of Earnings Prediction with Local LLMs

〇Shirai Yusuke¹, Ichikawa Yoshihiko¹, Nakagawa Kei² (1. Insight Edge.Inc, 2. Osaka Metropolitan University)

Keywords:

LLM,Open Weights LLM,Edinet,Earning Forecasting

We apply large language models (LLMs) to predict whether corporate profits in the Japanese stock market will increase or decrease. With practical use in financial institutions, we focus on local (on-premise/open weights) LLMs and evaluate their performance. Specifically, we use EDINET-BENCH, a benchmark dataset built from Japan’s Financial Services Agency EDINET filings, to assess how well local LLMs can predict the direction of profit changes. First, by separating each model’s pretraining cutoff date from the disclosure dates of annual securities reports, we distinguish in-sample from out-of-sample periods and measure the true generalization performance of local LLMs. Then, we examine how prediction accuracy varies with firm characteristics such as industry and firm size. Finally, we analyze the reasoning explanations generated by the LLMs to identify patterns of firms that each model tends to handle well or poorly.