[4K1-GS-6a] Language media processing

10:00 AM - 10:15 AM JST(1:00 AM - 1:15 AM UTC)

[4K1-GS-6a-05] Enhancing the Logical Reasoning Capabilities of Small-Scale Language Models

Yusei Kimura¹, Kazunori Komatani², 〇MASAYUKI NUMAO¹ (1. Kyoto Tachibana University, 2. Osaka University)

Keywords:

Large Language Models,Small-Scale Language Models,math word problem

Large Language Models (LLMs) demonstrate high performance, but they also involve significant computational resources, operational costs, and security concerns associated with using external servers. In contrast, small-scale language models (SLMs) designed to operate locally are easier to handle but often suffer from insufficient reasoning capabilities, resulting in low accuracy rates on logical tasks like arithmetic word problems. This research proposes a stepwise process design that divides problem-solving into multiple stages to improve the reasoning performance of such small-scale language models, and evaluates it using GSM8K. Specifically, the model first generates a brief solution strategy (what to compute), then performs the computation, and finally formulates the answer. Evaluation showed that the final accuracy rate increased slightly when using the staged approach compared to without it, but no significant difference was observed. However, observing the intermediate outputs in the staged case revealed that even for incorrect problems, the solution approach itself was often correct. Furthermore, while incorrect answers decreased with staging, unanswered questions increased. We speculate this may be because being shown the steps made it harder for the system to generate a suboptimal answer.