Presentation Information
[2H4-OS-2a-01]Selection Biases in Pairwise Financial Sentiment Analysis with Large Language Models
〇Kaito Takano1,2 (1. Nomura Asset Management Co., Ltd., 2. Osaka Metropolitan University)
Keywords:
Financial,LLMs,Sentiment Analysis,Pairwise,Selection Bias
With the advancement of Large Language Models (LLMs), they have increasingly been applied to sentiment analysis.
When analyzing subjective and difficult-to-quantify phenomena such as sentiment—especially when the goal is to evaluate relative strength or intensity—comparative approaches based on text-to-text evaluation, such as pairwise and listwise methods, are useful. While prior research has investigated biases in pointwise financial sentiment analysis, where the financial sentiment of a single text is assessed independently, bias analysis in pairwise financial sentiment analysis remains insufficient.
In this study, we examine what kinds of selection biases arise when financial sentiment analysis is conducted through pairwise evaluation using LLMs.
The results of our empirical analysis reveal that:
(1) there is a tendency to make selections by emphasizing specific factors, and
(2) company names influence the selection outcomes.
When analyzing subjective and difficult-to-quantify phenomena such as sentiment—especially when the goal is to evaluate relative strength or intensity—comparative approaches based on text-to-text evaluation, such as pairwise and listwise methods, are useful. While prior research has investigated biases in pointwise financial sentiment analysis, where the financial sentiment of a single text is assessed independently, bias analysis in pairwise financial sentiment analysis remains insufficient.
In this study, we examine what kinds of selection biases arise when financial sentiment analysis is conducted through pairwise evaluation using LLMs.
The results of our empirical analysis reveal that:
(1) there is a tendency to make selections by emphasizing specific factors, and
(2) company names influence the selection outcomes.