[1K5-GS-3c] Knowledge utilization and sharing

6:10 PM - 6:25 PM JST(9:10 AM - 9:25 AM UTC)

[1K5-GS-3c-03] Assumption Lens Framework: Diagnosing LLM/VLM Behavior Annotation via Implicit Assumption Mapping

〇YUKI YAMAGATA¹, Yuta Inaba¹, Teruhisa S Komatsu¹, Shuichi Onami¹, Hiroshi Masuya¹ (1. RIKEN)

Keywords:

Ontology,LLM,annotation,behavior analysis

Video behavior annotation with Large Language Models (LLMs) and Vision-Language Models (VLMs) is advancing toward practical implementation. This study addresses the ambiguity between "observation" and "inference" in generated descriptions. We propose the Assumption Lens Framework, based on an ontological approach, to structure these discrepancies as differences in implicit assumptions bridging observed facts and interpretive vocabulary. By mapping model-specific characteristics onto an assumption space, this framework enables comparative analysis. We report on an evaluation using mouse behavior storyboards, demonstrating that our method consistently diagnoses interpretive gaps and facilitates controllable re-annotation.