Presentation Information
[A-11-09]Evaluation of Automated Program Repair Methods Using LLMs with CoT Prompting
◎△Yuto Hiratsuka1, Keisuke Kitamura2, Akihito Kohiga1, Takahiro Koita2 (1. Doshisha University, 2. Graduate School of Doshisha University)
Keywords:
LLM,APR
Bug fixing is costly in software development, and Automated Program Repair (APR) has been actively studied to address this. Traditional template-based APR is limited in the bug patterns it can handle. Recent Large Language Models (LLMs) have shown strong performance in code understanding and generation, making them promising for flexible APR. However, prompt design directly affects repair performance, making prompt engineering critical. Chain-of-Thought (CoT) prompting encourages step-by-step reasoning and has been reported to improve accuracy on complex tasks. Since bug fixing requires logical reasoning to identify root causes, CoT is expected to be effective in APR. In this study, we apply CoT prompting to zero-shot LLM-based APR and evaluate its effect on repair success rate. Experiments used Math and Lang projects from Defects4J Version 3.0.1 (147 bugs) with OpenAI GPT-5 mini. The CoT condition achieved 28.6%, outperforming non-CoT (19.0%) by 9.6 percentage points. Math showed a larger improvement (+12.8%) than Lang (+3.8%), suggesting mathematical bugs benefit more from step-by-step reasoning. These results indicate that CoT prompting is promising for improving LLM-based APR.