[5L2-OS-6a] OS-6

12:30 PM - 12:45 PM JST(3:30 AM - 3:45 AM UTC)

[5L2-OS-6a-03] Proposal on Recommendation Item Decision Strategy Based on Pairwise Comparison by LLM

〇Tomohiro Tani¹, Hiroki Shibata¹, Yasufumi Takama¹ (1. Tokyo Metropolitan University)

Keywords:

Recommendation,LLM,Ranking

This paper proposes a zero-shot and scalable approach for generating recommendation lists with large language models (LLMs), in which pairwise preference judgments are aggregated to determine an item ranking.Although recent studies have explored leveraging the broad knowledge and advanced text understanding capabilities of LLMs in recommender systems, it is often impractical to have an LLM evaluate all candidate items at once when the number of candidates is large.To address this issue, the proposed method queries an LLM multiple times with separate prompts and then aggregates the resulting judgments. Specifically, the LLM is prompted to assess preference relations at the level of item pairs, and a recommendation list is constructed via Bradley–Terry–Luce (BTL) aggregation.The effectiveness of the proposed method is evaluated with experiments.Because the method relies on multiple LLM evaluations, inconsistent judgments may arise and potentially affect recommendation accuracy.The analysis of the occurrence of such inconsistencies shows that, while the proportion of inconsistencies increases as the number of item pairs grows, recommendation performance does not degrade significantly.