[3Yin-A] Poster 5

[3Yin-A-29] A Predictive State Representation View of Large Language Model GenerationPossibilities of a dynamical dystems Interpretation of LLMs

〇Shinnosuke Mizuno¹, Yohei Kobashi¹ (1. The University of Tokyo)

Keywords:

LLM,persona,word graph

Large language models (LLMs) generate responses that exhibit diverse personas even for the same query, depending on the prompt; however, methods for precise persona control remain underdeveloped. In this study, we define persona as a low-dimensional bias in the conditional next-token distribution given an input distribution, and propose a framework for its identification and comparison based on the concept of Predictive State Representation (PSR). Given a fixed set of concept nodes and relation labels, we estimate—via teacher forcing—the probability of each relation label for every directed node pair, and construct a persona graph in which these probabilities serve as edge attributes. Through experiments across multiple LLMs and varying generation conditions, we demonstrate that persona inputs induce consistent and reproducible differences in graph structure across models. Furthermore, we show that stable edges with low condition dependence enable visualization of the persona space and suggest the possibility of continuous transitions toward a target persona. We will refine this framework to develop techniques that enable rigorous and fine-grained steering toward arbitrary personas.