[2Yin-A] Poster 3

[2Yin-A-05] Accuracy Evaluation of AI Agent Architectures for Requirements Specification Review Tasks

〇Keiichi Mase¹, Takashi Orihara¹, Kuniharu Yamada¹ (1. NS Solutions Corporation, Systems Research & Development Center)

Keywords:

agent,supervisor,workflow

Reviewing requirements specification documents is a critical task in the early stages of system development, as it affects system quality and rework costs. However, checking non-functional requirements still relies heavily on manual effort and requires significant time and labor. Recently, generative AI has been applied to support this process, and multi-agent architectures have attracted attention. Typical approaches include supervisor-based architectures, which emphasize flexibility and extensibility, and workflow-based architectures, which follow predefined processing steps. Although supervisor-based architectures are often considered less accurate, this assumption has rarely been validated in practical tasks. In this study, we define a task in which generative AI checks whether non-functional requirements are described in requirements specification documents and generates review comments. We compare the accuracy of supervisor-based and workflow-based architectures. The results show that the supervisor-based architecture exhibits approximately 7% lower accuracy than the workflow-based architecture, indicating that workflow-based approaches are more suitable when accuracy is prioritized. Conversely, the supervisor-based architecture offers higher extensibility, making it useful when future functional expansion is required.