Presentation Information
[N-2-02]Classification Analysis of Contextual Hallucination Based on Low-dimensional Lookback Ratio Components
〇Yuto Sasaki1, Ken'ya Jin'no1 (1. Tokyo City Univ.)
Keywords:
Contextual Hallcination,Lookback Ratio,Large Language Model
Contextual Hallucination (CH), where a large language model generates content unsupported by the input context, is an important issue in summarization. Lookback Lens detects CH using Lookback Ratio (LR) features from attention maps over all layers and heads. This study examines whether CH-related information in 1024-dimensional LR features can be summarized as low-dimensional latent components. We generated CNN/DailyMail summaries with Phi-3-mini and assigned binary CH labels using a gpt-oss-based LLM judge. For each summary, length-normalized LR values averaged over output time steps were extracted as 1024-dimensional features. After training-set standardization, we analyzed inter-head correlations and eigenvalues of the correlation matrix. The mean absolute correlation was 0.285, 55.6% of head pairs had |r|>0.2, and the top 8 and 32 components explained 67.5% and 80.2% of the variance. We compared LR-1024, PCA-32, and PLS-8 with L2-regularized logistic regression. AUROC/AUPRC were 0.704/0.344, 0.714/0.341, and 0.722/0.338. The results suggest that CH classification information may appear as low-dimensional LR patterns across multiple heads.