Presentation Information
[5Yin-A-62]Performance Evaluation of Generative AI in Accounting and AuditingExperimental evidence from the Japanese CPA Auditing Examination
〇Yoshitaka Hirose1 (1. Doshisha University)
Keywords:
Auditing,Accountancy,CPA exam,co-pilot
In recent years, the rapid advancement of generative AI based on large language models (LLMs) has increased interest in the application of generative AI in the accounting and auditing industry. While reports indicate that generative AI has achieved passing scores on the U.S. CPA Examination, prior research evaluating generative AI performance on the Japanese CPA Examination—a highly competitive test with a pass rate of approximately 10%—remains limited. This study aims to compare and evaluate the response accuracy of multiple generative AI models, including ChatGPT, Gemini, and Claude, on the Auditing section of the Japanese CPA Short-Answer Examination, thereby clarifying the capabilities and limitations of generative AI in processing specialized knowledge in accounting and auditing. Specifically, each model was tested using publicly available past examination questions. The results showed that the latest models demonstrated substantially improved accuracy compared to earlier versions, with performance exceeding both the average scores of examinees and the passing threshold. These findings suggest the potential for integrating generative AI into audit practice and provide practical implications for collaboration between AI and certified public accountants.